-- Logs begin at Wed 2022-08-24 04:50:40 UTC, end at Wed 2022-08-24 06:13:18 UTC. -- Aug 24 05:22:43.482851 dv-cinder-854267-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 24 05:22:43.491457 dv-cinder-854267-1-iscsi devstack@keystone.service[81662]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 24 05:22:43.493690 dv-cinder-854267-1-iscsi devstack@keystone.service[81662]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 24 05:22:43 2022] *** Aug 24 05:22:43.493690 dv-cinder-854267-1-iscsi devstack@keystone.service[81662]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 24 05:22:43.493690 dv-cinder-854267-1-iscsi devstack@keystone.service[81662]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 24 05:22:43.493690 dv-cinder-854267-1-iscsi devstack@keystone.service[81662]: nodename: dv-cinder-854267-1-iscsi Aug 24 05:22:43.493690 dv-cinder-854267-1-iscsi devstack@keystone.service[81662]: machine: x86_64 Aug 24 05:22:43.493690 dv-cinder-854267-1-iscsi devstack@keystone.service[81662]: clock source: unix Aug 24 05:22:43.493690 dv-cinder-854267-1-iscsi devstack@keystone.service[81662]: pcre jit disabled Aug 24 05:22:43.493690 dv-cinder-854267-1-iscsi devstack@keystone.service[81662]: detected number of CPU cores: 4 Aug 24 05:22:43.493690 dv-cinder-854267-1-iscsi devstack@keystone.service[81662]: current working directory: / Aug 24 05:22:43.493690 dv-cinder-854267-1-iscsi devstack@keystone.service[81662]: detected binary path: /usr/bin/uwsgi-core Aug 24 05:22:43.493690 dv-cinder-854267-1-iscsi devstack@keystone.service[81662]: your processes number limit is 47680 Aug 24 05:22:43.494364 dv-cinder-854267-1-iscsi devstack@keystone.service[81662]: your memory page size is 4096 bytes Aug 24 05:22:43.494364 dv-cinder-854267-1-iscsi devstack@keystone.service[81662]: detected max file descriptor number: 2048 Aug 24 05:22:43.494364 dv-cinder-854267-1-iscsi devstack@keystone.service[81662]: lock engine: pthread robust mutexes Aug 24 05:22:43.494364 dv-cinder-854267-1-iscsi devstack@keystone.service[81662]: thunder lock: enabled Aug 24 05:22:43.494364 dv-cinder-854267-1-iscsi devstack@keystone.service[81662]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 24 05:22:43.494364 dv-cinder-854267-1-iscsi devstack@keystone.service[81662]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 24 05:22:43.537729 dv-cinder-854267-1-iscsi devstack@keystone.service[81662]: Python main interpreter initialized at 0x5579d2ece9c0 Aug 24 05:22:43.537729 dv-cinder-854267-1-iscsi devstack@keystone.service[81662]: python threads support enabled Aug 24 05:22:43.537729 dv-cinder-854267-1-iscsi devstack@keystone.service[81662]: your server socket listen backlog is limited to 100 connections Aug 24 05:22:43.537729 dv-cinder-854267-1-iscsi devstack@keystone.service[81662]: your mercy for graceful operations on workers is 90 seconds Aug 24 05:22:43.539601 dv-cinder-854267-1-iscsi devstack@keystone.service[81662]: mapped 403077 bytes (393 KB) for 2 cores Aug 24 05:22:43.539709 dv-cinder-854267-1-iscsi devstack@keystone.service[81662]: *** Operational MODE: preforking *** Aug 24 05:22:43.540032 dv-cinder-854267-1-iscsi devstack@keystone.service[81662]: *** uWSGI is running in multiple interpreter mode *** Aug 24 05:22:43.540032 dv-cinder-854267-1-iscsi devstack@keystone.service[81662]: spawned uWSGI master process (pid: 81662) Aug 24 05:22:43.540263 dv-cinder-854267-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 24 05:22:43.542638 dv-cinder-854267-1-iscsi devstack@keystone.service[81662]: spawned uWSGI worker 1 (pid: 81670, cores: 1) Aug 24 05:22:43.542638 dv-cinder-854267-1-iscsi devstack@keystone.service[81662]: spawned uWSGI worker 2 (pid: 81671, cores: 1) Aug 24 05:22:43.542638 dv-cinder-854267-1-iscsi devstack@keystone.service[81662]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 24 05:22:48.020802 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 05:22:48.020802 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: from cryptography.utils import int_from_bytes Aug 24 05:22:48.021663 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 05:22:48.021663 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: from cryptography.utils import int_from_bytes Aug 24 05:22:48.023750 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 05:22:48.023750 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: from cryptography.utils import int_from_bytes Aug 24 05:22:48.024209 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 05:22:48.024209 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: from cryptography.utils import int_from_bytes Aug 24 05:22:48.076249 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 05:22:48.076529 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 05:22:48.076632 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 05:22:48.076726 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 05:22:48.076810 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] command line args: [] {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 05:22:48.076892 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] command line args: [] {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 05:22:48.077004 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 05:22:48.077081 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 05:22:48.077151 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ================================================================================ {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 05:22:48.077222 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ================================================================================ {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 05:22:48.077298 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] admin_token = **** {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.077374 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] admin_token = **** {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.077524 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] config_dir = [] {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.077613 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] config_dir = [] {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.077718 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.077806 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.077904 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] config_source = [] {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.077995 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] config_source = [] {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.078081 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] debug = True {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.078160 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] debug = True {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.078270 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.079683 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.079829 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.079933 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.080038 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.080138 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.080235 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.080322 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.080449 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.080528 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.080682 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] list_limit = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.081090 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] list_limit = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.081186 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] log_config_append = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.081372 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] log_config_append = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.081523 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.081665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.081803 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] log_dir = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.081907 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] log_dir = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.081983 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] log_file = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.082105 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] log_file = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.082236 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.082466 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.082550 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.082624 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.082750 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.082828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.082958 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.083045 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.083121 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.083121 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.083321 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.083321 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.083517 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.083598 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.083718 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.083718 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.083897 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.084023 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.084102 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.084162 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.084278 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.084370 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.084529 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.084616 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.084679 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.084805 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.084888 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.084960 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.085042 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.085042 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.085178 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.085178 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] publish_errors = False {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.085389 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.085471 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.085548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] publish_errors = False {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.085625 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.085704 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.085785 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.085910 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.085993 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.086065 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.086151 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.086234 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.086312 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] standard_threads = False {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.086383 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.086499 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.086575 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] standard_threads = False {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.086654 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.086726 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.086804 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.086876 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.086952 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] use_journal = False {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.087029 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.087111 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] use_json = False {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.087189 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] use_journal = False {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.087261 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] use_stderr = False {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.087330 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] use_json = False {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.087400 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] use_syslog = False {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.087469 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] use_stderr = False {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.087729 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.087814 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] use_syslog = False {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.087875 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.087994 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:22:48.088071 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.088131 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.088204 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.088325 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.088418 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.088491 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.088561 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.088630 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.088714 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.088794 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.088872 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.088938 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.089003 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.089067 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.089134 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.089245 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.089326 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.089443 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.089532 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.089697 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.089787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.089951 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.090040 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.090190 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.090268 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.090440 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.090523 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.090691 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.090771 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.090914 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.090991 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.091167 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.091247 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.091409 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.091495 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.091652 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.091736 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.091890 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.091970 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.092144 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.092221 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.092381 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.092557 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.092715 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.092792 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.092935 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.093018 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.093123 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.093199 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.093355 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.093462 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.093584 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.093698 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.093821 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.093934 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.094071 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.094151 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.094317 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.094396 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.094547 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.094637 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.094778 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.094864 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.095024 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.095113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.095270 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.095356 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.095507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.095594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.095760 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.095838 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.095990 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.096099 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.096228 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.096343 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.096545 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.096703 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.096917 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.097102 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.097298 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.097456 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.097567 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.097650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.097770 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.097856 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.097963 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.098062 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.098142 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.098261 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.098337 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.098418 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.098489 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.098551 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.098621 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.098699 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.098771 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.098831 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.098960 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.099039 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.099234 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.099312 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.099438 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.099516 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.099676 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.099754 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.099912 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.099982 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.100148 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.100216 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.100385 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.100489 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.100644 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.100752 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.100875 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.100944 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.101124 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.101195 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.101357 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.101426 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.101591 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.101663 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.101821 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.101894 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.102081 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.102151 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.102317 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.102387 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.102547 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.102627 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.102782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.102858 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.103016 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.103093 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.103282 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.103383 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.103513 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.103599 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.103759 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.103840 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.104003 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.104158 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.104279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.104361 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.104501 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.104577 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.104708 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.108136 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.108258 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.108347 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.108444 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.108520 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.108592 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.108688 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.108775 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.108856 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.108929 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.109000 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.109071 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.109141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.109223 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.109298 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.109375 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.109449 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.109520 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.109596 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.109678 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.109752 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.109824 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.109894 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.109965 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.110036 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.110115 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.110194 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.110319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.110404 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.110468 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.110586 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.110677 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.110677 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.110860 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.110938 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.111010 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.111081 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.111161 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.111235 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.111306 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.111383 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.111463 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.111535 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.111663 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.111743 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.111816 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.111892 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.111973 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.112595 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.112682 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.112788 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.112920 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.113031 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.113151 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.113282 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.113436 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.113570 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.113667 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.113741 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] auth.external = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.113813 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.113884 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.113965 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.114092 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.114172 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.114244 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.114315 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] auth.external = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.114315 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.114487 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] auth.password = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.114561 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.114691 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] auth.token = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.114691 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.114836 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.114910 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.114982 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] auth.password = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.115054 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.115133 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] auth.token = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.115256 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.115334 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.115462 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.115552 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.115628 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.115709 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.115791 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.115866 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.115939 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] credential.caching = True {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.116010 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.116194 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.116282 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.116429 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.116511 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.116585 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.116585 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.116730 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] credential.caching = True {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.116814 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.116889 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.116961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.117033 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.117105 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.117185 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.117260 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.117332 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.117409 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.117483 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.117555 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.117635 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.117842 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.117924 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.118088 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.118176 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.118329 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.118487 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.118565 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.118643 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.118819 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.118901 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.119053 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] federation.caching = True {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.119143 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.119287 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.119369 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.119562 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.119653 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.119773 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.119910 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.119993 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.120154 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] federation.caching = True {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.120229 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.120421 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.120506 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.120678 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.120761 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.120929 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.121017 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.121167 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.121249 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.121404 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.121485 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.121649 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.121729 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.121884 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] identity.caching = True {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.122048 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.122148 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.122226 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.122358 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.122440 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.122591 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.122672 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.122824 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.122907 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.123068 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.123155 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] identity.caching = True {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.123298 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.123383 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.123534 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.123616 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.123780 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.123861 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.124020 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.124100 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.124248 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.124329 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.124501 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.124586 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.124732 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.124813 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.124974 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.125057 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.125219 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.125513 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.125513 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.125791 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.125862 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.125992 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.126073 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.126296 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.126383 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.126468 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.126597 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.126676 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.126749 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.126822 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.126893 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.126974 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.127082 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.127171 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.127311 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.127393 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.127546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.127686 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.128366 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.128541 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.128667 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.128877 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.129040 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.129182 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.129267 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.129341 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.129509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.129592 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.129754 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.129844 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.130003 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.130084 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.130245 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.130325 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.130476 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.130651 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.130733 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.130806 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.130927 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.131008 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.131163 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.131252 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.131399 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.131480 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.131658 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.131784 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.131911 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.132001 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.132150 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.132228 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.132445 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.132532 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.132655 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.132755 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.132901 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.132983 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.133144 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.133225 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.133415 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.133545 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.133663 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.133792 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.133900 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.134036 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.134123 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.134294 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.134377 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.134548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.134628 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.user = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.134786 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.134873 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.135024 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.135106 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.135259 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.135339 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.135507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.135587 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.135753 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.135836 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.135998 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.136079 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.136238 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.136318 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.136501 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.136583 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.user = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.136717 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.136799 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.136954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.137035 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.137190 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.137273 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.137420 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.137500 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.137666 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.137746 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.137908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.137991 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.138152 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.138234 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.138392 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.138472 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.138635 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.138715 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.138865 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.138936 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.139113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.139193 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.139347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.139429 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.139583 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.139654 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.139817 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.139888 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.140050 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.140122 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.140293 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.140364 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.140567 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.140665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.140802 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.140882 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.141055 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.141136 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.141298 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.141392 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.141539 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.141620 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.141779 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.141860 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.142018 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.142173 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.142253 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.142326 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.142506 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.142588 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.142746 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.142859 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.143008 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.143126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.143269 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.143539 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.143660 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.143850 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.143996 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] resource.caching = True {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.144135 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.144321 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.144424 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.144598 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.144842 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.144978 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.145120 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.145202 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.145306 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.145388 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.145552 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] resource.caching = True {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.145635 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.145806 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.145887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.146060 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.146140 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.146301 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.146385 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.146550 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.146630 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.146786 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] role.caching = True {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.146867 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.147030 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.147112 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] role.driver = sql {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.147261 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.147349 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.147508 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.147588 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.147753 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.147835 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.147992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] role.caching = True {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.148073 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.148237 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.148317 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] role.driver = sql {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.148495 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.148581 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.148749 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.148829 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.148996 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.149077 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.149243 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.149326 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.149475 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.149556 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.149721 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.149801 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.149966 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.150049 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.150211 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.150290 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.150454 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.150534 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.150688 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.150891 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.150977 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.151132 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.151211 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.151363 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.151446 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.151611 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.151693 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.151857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.152065 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.152154 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.152305 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.152387 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.152584 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.152666 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.152834 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.158467 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.158678 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.158776 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.158885 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.158988 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.159085 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.159225 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.159348 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.159478 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.159614 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.159755 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.159875 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.159956 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.160019 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.160110 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.160209 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.160291 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.160366 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.160473 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.160551 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.160624 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.160697 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] token.caching = True {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.160769 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.160840 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.160921 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.160995 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.161068 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.161141 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.161213 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.161285 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.161366 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.161441 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.161514 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] token.caching = True {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.161628 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.161716 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.161791 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.161873 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.161947 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.162009 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.162068 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.162127 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.162186 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.162253 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.162315 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.162375 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.162433 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.162505 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.162581 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.162655 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.162718 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.162785 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.162785 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.162894 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.162953 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.163020 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.163082 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.163141 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.163199 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.163257 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] database.connection = **** {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.163328 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.163397 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.163465 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.163526 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.163585 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.163643 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.163702 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.163760 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.163827 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] database.connection = **** {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.163897 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.163966 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.164040 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.164112 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.164173 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.164173 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.164357 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.164506 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.164584 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.164657 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.164730 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.164802 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.164877 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.164941 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.165001 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.165060 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.165119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.165178 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.165245 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.165308 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.165367 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.165427 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.165486 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.165545 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.165619 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.165681 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.165755 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.165827 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81670) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 05:22:48.165889 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.165889 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.165889 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.165889 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.165889 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:22:48.165889 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81671) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 05:22:48.195629 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81670) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.195859 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81670) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.195952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81671) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.196078 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81670) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.196167 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81671) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.196487 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81671) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.204923 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81670) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.205344 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81671) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.233443 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81670) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.233671 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81670) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.234042 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81671) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.234280 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81671) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.289680 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81670) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.290779 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81671) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.340553 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81670) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.340751 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81671) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.340825 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81670) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.340918 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81671) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.510233 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81670) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.510463 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81670) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.510694 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81670) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.510920 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81670) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.511143 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81670) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.511362 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81670) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.511939 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81671) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.512176 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81671) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.512436 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81671) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.512655 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81671) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.512877 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81671) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.513095 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81671) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.536703 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81671) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.536931 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81671) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.537074 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81670) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.537162 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81671) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.537273 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81670) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.537358 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81671) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.537501 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81670) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.537587 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81671) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.537764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81670) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.537869 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81671) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.537946 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81670) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.538020 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81671) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.538153 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81670) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.538372 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81670) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.549586 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81671) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.550348 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81670) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.575760 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81671) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.575939 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81670) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.576051 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81671) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.576135 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81670) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.576211 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81671) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.576287 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81670) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.576423 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81671) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.576540 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81670) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.576705 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81671) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.576806 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81670) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.576884 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81671) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.576958 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81670) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.577091 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81671) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.577211 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81670) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:22:48.594992 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.595164 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.595256 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 05:22:48.595339 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 05:22:48.595543 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.595642 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.595731 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 05:22:48.595808 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 05:22:48.596046 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.596133 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.596313 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.596474 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.596984 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.597087 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.597253 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.609549 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.609681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.609753 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.609830 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.609830 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.609969 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.610046 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.610123 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.610123 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.614708 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81671) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:22:48.615441 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81670) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:22:48.617701 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81671) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:22:48.617992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.618280 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.618440 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81670) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:22:48.618556 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.618737 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.618828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.619030 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.619136 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.619323 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.619408 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.619611 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.619694 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.619873 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.620008 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.620153 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.620284 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.620450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.620591 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.620878 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.621215 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.621518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.644708 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.645881 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.646220 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81671) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:22:48.646506 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.647434 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81670) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:22:48.647726 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.651278 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81671) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:22:48.651566 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.651850 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.652493 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81670) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:22:48.652785 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.653072 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.658537 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81671) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:22:48.658847 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.660037 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81670) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:22:48.660331 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.663207 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81671) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:22:48.663499 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.663787 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.664072 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.664354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.664524 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81670) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:22:48.664759 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.664856 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.665002 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.665183 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.665287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.665478 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.665759 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.666037 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.666312 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.666585 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.683361 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.683654 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.685051 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.685352 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.687573 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81671) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:22:48.693943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81670) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:22:48.694067 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.694067 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.694853 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.694853 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.695976 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81671) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:22:48.698427 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81670) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:22:48.699174 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81671) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:22:48.701610 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81670) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:22:48.702450 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.702755 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.703047 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.703333 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.703610 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.703900 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.705001 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.705309 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.705600 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.705966 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.706257 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.706557 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.719874 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81671) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:22:48.720169 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.720482 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.720729 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.723172 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81670) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:22:48.723476 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.723755 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.724023 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.727787 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.729354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.729634 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.731175 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.732270 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81671) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:22:48.732589 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.732772 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.732937 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.733094 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.733244 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.733522 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.735827 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81670) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:22:48.736123 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.736437 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.736751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.737576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.745539 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81671) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:22:48.745830 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.746114 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.746397 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.746681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.746978 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.747268 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.749735 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81670) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:22:48.750027 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.750353 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.750738 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.751025 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.751332 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.751632 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.763780 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81671) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:22:48.766889 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81671) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:22:48.768663 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81670) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:22:48.770030 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.771473 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.771883 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81670) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:22:48.772983 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81671) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:22:48.773286 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.773586 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.774997 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.776484 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.777972 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81670) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:22:48.778268 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.778596 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.780390 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.781891 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81671) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:22:48.785052 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.785345 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.785640 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.785931 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.786068 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.787676 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81670) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:22:48.791011 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.791332 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.791698 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.791995 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.793927 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81671) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:22:48.794226 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.794514 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.800707 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81670) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:22:48.801097 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.801388 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.802184 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81671) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:22:48.802474 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.802759 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.803037 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.803309 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.803580 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.803853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.804136 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.804435 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.804717 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.804988 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.805351 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.805527 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.805797 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81671) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.808417 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81670) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:22:48.808710 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.808997 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.809276 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.809562 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.809847 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.810127 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.810399 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.810690 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.810965 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.811254 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.811520 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.811798 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.812094 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81670) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:22:48.841164 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5579d2ece9c0 pid: 81671 (default app) Aug 24 05:22:48.844748 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e94b9-6e49-4c94-ad99-0a0cdb353c04 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:22:48.845048 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e94b9-6e49-4c94-ad99-0a0cdb353c04 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:22:48.845325 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e94b9-6e49-4c94-ad99-0a0cdb353c04 None None] PATH_INFO: `/v3` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:22:48.848133 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5579d2ece9c0 pid: 81670 (default app) Aug 24 05:22:48.850365 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1/1] 10.222.0.15 () {50 vars in 791 bytes} [Wed Aug 24 05:22:48 2022] GET /identity/v3/ => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:22:57.203396 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561e800-f562-4321-bc5e-c65bd5785caf None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:22:57.203950 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561e800-f562-4321-bc5e-c65bd5785caf None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:22:57.203950 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561e800-f562-4321-bc5e-c65bd5785caf None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:22:57.208924 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1/2] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:22:57 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:22:57.215859 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc12dcb-885d-468c-9d6a-56b5a42a9751 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:22:57.216368 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc12dcb-885d-468c-9d6a-56b5a42a9751 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:22:57.216500 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc12dcb-885d-468c-9d6a-56b5a42a9751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:22:57.292290 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: INFO dbcounter [None req-ecc12dcb-885d-468c-9d6a-56b5a42a9751 None None] Registered counter for database keystone Aug 24 05:22:57.315394 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG oslo_db.sqlalchemy.engines [None req-ecc12dcb-885d-468c-9d6a-56b5a42a9751 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81671) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 05:22:57.321863 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writer thread running {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 05:22:57.505768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG passlib.handlers.bcrypt [None req-ecc12dcb-885d-468c-9d6a-56b5a42a9751 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81671) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 05:22:57.506102 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG passlib.handlers.bcrypt [None req-ecc12dcb-885d-468c-9d6a-56b5a42a9751 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81671) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 05:22:57.562093 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-ecc12dcb-885d-468c-9d6a-56b5a42a9751 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:22:57.563704 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-ecc12dcb-885d-468c-9d6a-56b5a42a9751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:22:57.705035 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 2/3] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:22:57 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 490 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 05:22:58.200054 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9561e800-f562-4321-bc5e-c65bd5785caf None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:22:58.270409 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: INFO dbcounter [None req-9561e800-f562-4321-bc5e-c65bd5785caf None None] Registered counter for database keystone Aug 24 05:22:58.290957 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG oslo_db.sqlalchemy.engines [None req-9561e800-f562-4321-bc5e-c65bd5785caf None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81670) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 05:22:58.297546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writer thread running {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 05:22:58.425405 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9561e800-f562-4321-bc5e-c65bd5785caf None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:22:58.429685 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5373c3-45cc-4eb8-aa9d-0c7a651db064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:22:58.430329 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5373c3-45cc-4eb8-aa9d-0c7a651db064 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:22:58.430609 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5373c3-45cc-4eb8-aa9d-0c7a651db064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:22:58.430882 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5373c3-45cc-4eb8-aa9d-0c7a651db064 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:22:58.439195 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5373c3-45cc-4eb8-aa9d-0c7a651db064 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:22:58.491113 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5c5373c3-45cc-4eb8-aa9d-0c7a651db064 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 05:22:58.491451 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5373c3-45cc-4eb8-aa9d-0c7a651db064 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:22:58.501635 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 2/4] 10.222.0.15 () {58 vars in 1324 bytes} [Wed Aug 24 05:22:58 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 05:22:58.507706 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc12dcb-885d-468c-9d6a-56b5a42a9751 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:22:58.523313 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc12dcb-885d-468c-9d6a-56b5a42a9751 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:22:58.527894 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad429a4c-fca2-484f-8aff-dd1866f80445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:22:58.529029 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad429a4c-fca2-484f-8aff-dd1866f80445 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:22:58.529314 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad429a4c-fca2-484f-8aff-dd1866f80445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:22:58.529588 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad429a4c-fca2-484f-8aff-dd1866f80445 admin admin] PATH_INFO: `/v3/projects/617e47339cb94e20967707fe62db290d` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:22:58.531520 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad429a4c-fca2-484f-8aff-dd1866f80445 admin admin] RBAC: Authorizing `identity:get_project(project_id=617e47339cb94e20967707fe62db290d)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:22:58.584926 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ad429a4c-fca2-484f-8aff-dd1866f80445 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 05:22:58.585270 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad429a4c-fca2-484f-8aff-dd1866f80445 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:22:58.589836 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 3/5] 10.222.0.15 () {56 vars in 1207 bytes} [Wed Aug 24 05:22:58 2022] GET /identity/v3/projects/617e47339cb94e20967707fe62db290d => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:00.543532 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60ac2c-136b-4d8e-b72a-112db84bb126 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:00.544030 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60ac2c-136b-4d8e-b72a-112db84bb126 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:00.544030 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60ac2c-136b-4d8e-b72a-112db84bb126 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:00.548145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 3/6] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:00.554880 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5a8a7-b401-4eb7-b51e-3c161255c308 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:00.555344 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5a8a7-b401-4eb7-b51e-3c161255c308 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:00.555439 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5a8a7-b401-4eb7-b51e-3c161255c308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:00.576143 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-60c5a8a7-b401-4eb7-b51e-3c161255c308 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:00.577721 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-60c5a8a7-b401-4eb7-b51e-3c161255c308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:00.587056 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 4/7] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:00 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 05:23:00.854857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60ac2c-136b-4d8e-b72a-112db84bb126 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:00.868103 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60ac2c-136b-4d8e-b72a-112db84bb126 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:00.872504 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2f4181-1a6a-465c-86b9-820de6d0d3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:00.873118 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f4181-1a6a-465c-86b9-820de6d0d3fb admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:00.873399 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f4181-1a6a-465c-86b9-820de6d0d3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:00.873674 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f4181-1a6a-465c-86b9-820de6d0d3fb admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:00.882791 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2f4181-1a6a-465c-86b9-820de6d0d3fb admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:00.884621 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2f4181-1a6a-465c-86b9-820de6d0d3fb admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:00.893918 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 4/8] 10.222.0.15 () {58 vars in 1324 bytes} [Wed Aug 24 05:23:00 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 05:23:00.900576 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5a8a7-b401-4eb7-b51e-3c161255c308 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:00.911004 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5a8a7-b401-4eb7-b51e-3c161255c308 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:00.915442 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884270ce-25e7-4362-9604-d211126cd0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:00.916083 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884270ce-25e7-4362-9604-d211126cd0e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:00.916359 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884270ce-25e7-4362-9604-d211126cd0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:00.916666 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884270ce-25e7-4362-9604-d211126cd0e9 admin admin] PATH_INFO: `/v3/users/359799b7d001453b90d984a05215cce6` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:00.917984 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884270ce-25e7-4362-9604-d211126cd0e9 admin admin] RBAC: Authorizing `identity:get_user(user_id=359799b7d001453b90d984a05215cce6)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:00.919802 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884270ce-25e7-4362-9604-d211126cd0e9 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:00.921402 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 5/9] 10.222.0.15 () {56 vars in 1198 bytes} [Wed Aug 24 05:23:00 2022] GET /identity/v3/users/359799b7d001453b90d984a05215cce6 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:02.956894 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666fc8a-0092-4e55-ab02-7e59e3a00b75 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:02.957425 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666fc8a-0092-4e55-ab02-7e59e3a00b75 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:02.957425 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666fc8a-0092-4e55-ab02-7e59e3a00b75 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:02.958837 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 5/10] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:02.964593 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290ac94-4cee-4020-a890-a7ebc3d8d389 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:02.965115 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290ac94-4cee-4020-a890-a7ebc3d8d389 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:02.965115 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290ac94-4cee-4020-a890-a7ebc3d8d389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:02.983711 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-5290ac94-4cee-4020-a890-a7ebc3d8d389 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:02.984986 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-5290ac94-4cee-4020-a890-a7ebc3d8d389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:02.992824 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77c81a-5354-47af-bb66-ca71088243b2 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:02.993117 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77c81a-5354-47af-bb66-ca71088243b2 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:02.993477 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77c81a-5354-47af-bb66-ca71088243b2 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:02.994798 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 6/11] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:23:03.000313 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8fa15-499b-4577-a025-91f4cf377a40 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:03.000636 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8fa15-499b-4577-a025-91f4cf377a40 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:03.000921 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8fa15-499b-4577-a025-91f4cf377a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:03.045687 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG passlib.handlers.bcrypt [None req-7af8fa15-499b-4577-a025-91f4cf377a40 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81670) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 05:23:03.046113 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG passlib.handlers.bcrypt [None req-7af8fa15-499b-4577-a025-91f4cf377a40 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81670) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 05:23:03.074047 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 6/12] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:03.092166 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-7af8fa15-499b-4577-a025-91f4cf377a40 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:03.093706 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-7af8fa15-499b-4577-a025-91f4cf377a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:03.195631 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 7/13] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:23:03.342307 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5290ac94-4cee-4020-a890-a7ebc3d8d389 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:03.367221 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:03.450395 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5290ac94-4cee-4020-a890-a7ebc3d8d389 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:03.463236 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af8fa15-499b-4577-a025-91f4cf377a40 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:03.486203 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103b970-9c78-4fc8-989f-325bfa4a3822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:03.486936 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103b970-9c78-4fc8-989f-325bfa4a3822 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:03.487220 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103b970-9c78-4fc8-989f-325bfa4a3822 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:03.487497 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103b970-9c78-4fc8-989f-325bfa4a3822 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:03.500145 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6103b970-9c78-4fc8-989f-325bfa4a3822 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:23:03.509787 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-6103b970-9c78-4fc8-989f-325bfa4a3822 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:23:03.509787 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:23:03.509787 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:23:03.509787 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:23:03.509787 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:23:03.509787 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:23:03.509787 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:23:03.509787 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:23:03.509787 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:23:03.509787 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:23:03.509787 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:23:03.509787 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:23:03.509787 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:23:03.509787 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:23:03.509787 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:23:03.509787 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:23:03.510773 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:23:03.510773 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:23:03.510773 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:23:03.510773 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:23:03.510773 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:23:03.510773 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:23:03.510773 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:23:03.510773 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:23:03.510773 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:23:03.510773 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:23:03.510773 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:23:03.510773 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:23:03.510773 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:23:03.510773 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:23:03.510773 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:23:03.510773 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:23:03.511659 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:23:03.511659 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:23:03.511659 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:23:03.511659 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:23:03.511659 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:23:03.511659 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:23:03.511659 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:23:03.511659 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:23:03.511659 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:23:03.511659 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:23:03.511659 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:23:03.511659 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:23:03.511659 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 7/14] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Aug 24 05:23:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:23:03.517063 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103b970-9c78-4fc8-989f-325bfa4a3822 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:03.579383 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af8fa15-499b-4577-a025-91f4cf377a40 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:03.582384 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:03.614458 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:03.618107 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff0e61-cab8-4fff-b0dd-912b8ce076e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:03.619020 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff0e61-cab8-4fff-b0dd-912b8ce076e2 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:03.619305 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff0e61-cab8-4fff-b0dd-912b8ce076e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:03.619581 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff0e61-cab8-4fff-b0dd-912b8ce076e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:03.623676 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103b970-9c78-4fc8-989f-325bfa4a3822 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:03.660185 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558955c0-0119-4038-939f-c3be686597d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:03.660878 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558955c0-0119-4038-939f-c3be686597d7 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:03.661165 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558955c0-0119-4038-939f-c3be686597d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:03.661444 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558955c0-0119-4038-939f-c3be686597d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:03.662129 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558955c0-0119-4038-939f-c3be686597d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:03.662434 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558955c0-0119-4038-939f-c3be686597d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:03.663838 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558955c0-0119-4038-939f-c3be686597d7 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:03.677312 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 8/15] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:23:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:23:03.683156 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558955c0-0119-4038-939f-c3be686597d7 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:03.696551 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ff0e61-cab8-4fff-b0dd-912b8ce076e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:03.698114 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ff0e61-cab8-4fff-b0dd-912b8ce076e2 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:03.785234 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558955c0-0119-4038-939f-c3be686597d7 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:03.799568 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 8/16] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:23:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 05:23:03.805813 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff0e61-cab8-4fff-b0dd-912b8ce076e2 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:03.808734 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:03.822682 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce07ee9-3185-4be4-b77a-ee8e83aa375e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:03.826397 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce07ee9-3185-4be4-b77a-ee8e83aa375e None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:03.826397 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce07ee9-3185-4be4-b77a-ee8e83aa375e None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:03.826397 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce07ee9-3185-4be4-b77a-ee8e83aa375e None admin] PATH_INFO: `/v3/users/359799b7d001453b90d984a05215cce6` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:03.826397 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce07ee9-3185-4be4-b77a-ee8e83aa375e None admin] RBAC: Authorizing `identity:get_user(user_id=359799b7d001453b90d984a05215cce6)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:03.826733 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce07ee9-3185-4be4-b77a-ee8e83aa375e None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:03.828227 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 9/17] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:23:03 2022] GET /identity/v3/users/359799b7d001453b90d984a05215cce6 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:23:03.833877 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce07ee9-3185-4be4-b77a-ee8e83aa375e None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:03.848698 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:03.910475 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff0e61-cab8-4fff-b0dd-912b8ce076e2 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:03.944162 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce07ee9-3185-4be4-b77a-ee8e83aa375e None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:03.947485 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13d255b-3a45-44c4-b0b5-af87e13f3288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:03.948354 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13d255b-3a45-44c4-b0b5-af87e13f3288 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:03.948674 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13d255b-3a45-44c4-b0b5-af87e13f3288 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:03.948954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13d255b-3a45-44c4-b0b5-af87e13f3288 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:03.969092 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13d255b-3a45-44c4-b0b5-af87e13f3288 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:03.970581 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13d255b-3a45-44c4-b0b5-af87e13f3288 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:03.980632 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d60744-8013-46eb-b1b5-452c1c3a1776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:03.981275 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d60744-8013-46eb-b1b5-452c1c3a1776 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:03.981551 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d60744-8013-46eb-b1b5-452c1c3a1776 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:03.981823 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d60744-8013-46eb-b1b5-452c1c3a1776 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:03.982966 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d60744-8013-46eb-b1b5-452c1c3a1776 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:03.984382 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d60744-8013-46eb-b1b5-452c1c3a1776 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:03.985959 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 10/18] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:23:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:23:03.990371 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-e13d255b-3a45-44c4-b0b5-af87e13f3288 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:23:03.990371 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:23:03.990371 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:23:03.990371 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:23:03.990371 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:23:03.990371 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:23:03.990371 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:23:03.990371 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:23:03.990371 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:23:03.990371 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:23:03.990371 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:23:03.990371 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:23:03.990371 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:23:03.990371 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:23:03.990371 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:23:03.990371 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:23:03.991668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:23:03.991668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:23:03.991668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:23:03.991668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:23:03.991668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:23:03.991668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:23:03.991668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:23:03.991668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:23:03.991668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:23:03.991668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:23:03.991668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:23:03.991668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:23:03.991668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:23:03.991668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:23:03.991668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:23:03.991668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:23:03.991668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:23:03.992503 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:23:03.992503 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:23:03.992503 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:23:03.992503 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 9/19] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:23:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:23:03.992708 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d60744-8013-46eb-b1b5-452c1c3a1776 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:03.997200 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13d255b-3a45-44c4-b0b5-af87e13f3288 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:04.058028 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:04.076980 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:04.096541 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d60744-8013-46eb-b1b5-452c1c3a1776 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:04.103407 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13d255b-3a45-44c4-b0b5-af87e13f3288 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:04.131591 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1341dc-925a-4c48-851a-cb1905cf472f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:04.132205 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1341dc-925a-4c48-851a-cb1905cf472f None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:04.132518 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1341dc-925a-4c48-851a-cb1905cf472f None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:04.132800 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1341dc-925a-4c48-851a-cb1905cf472f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:04.133536 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1341dc-925a-4c48-851a-cb1905cf472f None admin] RBAC: Adding query filter params (role.id=e013ad4d436f4cb0a9bf7debb7d13dc7, scope.domain.id=default, user.id=359799b7d001453b90d984a05215cce6) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:04.133849 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1341dc-925a-4c48-851a-cb1905cf472f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:04.135292 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1341dc-925a-4c48-851a-cb1905cf472f None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:04.139036 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762fa558-4614-4856-a486-7986e09bef45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:04.139733 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fa558-4614-4856-a486-7986e09bef45 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:04.139948 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fa558-4614-4856-a486-7986e09bef45 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:04.140227 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fa558-4614-4856-a486-7986e09bef45 None admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:04.140985 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762fa558-4614-4856-a486-7986e09bef45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:04.141420 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762fa558-4614-4856-a486-7986e09bef45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:04.142272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 11/20] 10.222.0.15 () {56 vars in 1322 bytes} [Wed Aug 24 05:23:03 2022] GET /identity/v3/role_assignments?user.id=359799b7d001453b90d984a05215cce6&scope.domain.id=default&role.id=e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 233 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:23:04.143088 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762fa558-4614-4856-a486-7986e09bef45 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:04.164246 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 10/21] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:23:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:23:06.472478 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b2479-8aaa-485b-8cc7-ded53777a029 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:06.473585 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b2479-8aaa-485b-8cc7-ded53777a029 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:06.474296 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b2479-8aaa-485b-8cc7-ded53777a029 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:06.476316 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 12/22] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:06.482746 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae31989-c21f-4e21-8c64-1e5e60962c26 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:06.483425 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae31989-c21f-4e21-8c64-1e5e60962c26 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:06.483425 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae31989-c21f-4e21-8c64-1e5e60962c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:06.515000 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-3ae31989-c21f-4e21-8c64-1e5e60962c26 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:06.516297 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-3ae31989-c21f-4e21-8c64-1e5e60962c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:06.529540 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90effe15-5157-4ba0-a991-8bcf6a0f29b8 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:06.530399 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90effe15-5157-4ba0-a991-8bcf6a0f29b8 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:06.530933 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90effe15-5157-4ba0-a991-8bcf6a0f29b8 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:06.532922 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 13/23] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:06.538995 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe286b-67c0-4446-b9f2-b30135ba2f72 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:06.539581 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe286b-67c0-4446-b9f2-b30135ba2f72 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:06.540070 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe286b-67c0-4446-b9f2-b30135ba2f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:06.577555 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-e0fe286b-67c0-4446-b9f2-b30135ba2f72 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:06.579584 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-e0fe286b-67c0-4446-b9f2-b30135ba2f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:06.710023 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 11/24] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:23:06.750652 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac0559-9169-4194-8f8d-73a47deca49e None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:06.750887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac0559-9169-4194-8f8d-73a47deca49e None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:06.751219 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac0559-9169-4194-8f8d-73a47deca49e None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:06.754342 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 12/25] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:06.761980 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d96da-1df0-4241-a284-f5350c28126e None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:06.762565 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d96da-1df0-4241-a284-f5350c28126e None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:06.763058 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d96da-1df0-4241-a284-f5350c28126e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:06.772812 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 14/26] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:06.801551 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-b49d96da-1df0-4241-a284-f5350c28126e None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:06.803163 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-b49d96da-1df0-4241-a284-f5350c28126e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:06.885102 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:06.919138 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 13/27] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:07.186082 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe286b-67c0-4446-b9f2-b30135ba2f72 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:07.291547 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d96da-1df0-4241-a284-f5350c28126e None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:07.306596 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:07.379341 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe286b-67c0-4446-b9f2-b30135ba2f72 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:07.466381 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c776779-eb0f-4fd1-b779-57b798484c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:07.467036 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c776779-eb0f-4fd1-b779-57b798484c75 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:07.467321 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c776779-eb0f-4fd1-b779-57b798484c75 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:07.467598 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c776779-eb0f-4fd1-b779-57b798484c75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:07.491243 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c776779-eb0f-4fd1-b779-57b798484c75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:07.493815 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c776779-eb0f-4fd1-b779-57b798484c75 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:07.514687 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d96da-1df0-4241-a284-f5350c28126e None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:07.535254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-6c776779-eb0f-4fd1-b779-57b798484c75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:23:07.535254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:23:07.535254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:23:07.535254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:23:07.535254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:23:07.535254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:23:07.535254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:23:07.535254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:23:07.535254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:23:07.535254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:23:07.535254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:23:07.535254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:23:07.535254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:23:07.535254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:23:07.535254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:23:07.535254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:23:07.536426 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:23:07.536426 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:23:07.536426 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:23:07.536426 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:23:07.536426 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:23:07.536426 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:23:07.536426 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:23:07.536426 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:23:07.536426 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:23:07.536426 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:23:07.536426 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:23:07.536426 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:23:07.536426 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:23:07.536426 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:23:07.536426 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:23:07.536426 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:23:07.536426 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:23:07.537366 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:23:07.537366 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:23:07.537366 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:23:07.537366 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 15/28] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:23:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:23:07.537366 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c776779-eb0f-4fd1-b779-57b798484c75 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:07.583066 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41f85ce-cdf5-49ff-ae91-cdc5e326dba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:07.583724 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f85ce-cdf5-49ff-ae91-cdc5e326dba9 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:07.584007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f85ce-cdf5-49ff-ae91-cdc5e326dba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:07.584420 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f85ce-cdf5-49ff-ae91-cdc5e326dba9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:07.601672 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41f85ce-cdf5-49ff-ae91-cdc5e326dba9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:23:07.608339 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-a41f85ce-cdf5-49ff-ae91-cdc5e326dba9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:23:07.608339 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:23:07.608339 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:23:07.608339 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:23:07.608339 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:23:07.608339 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:23:07.608339 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:23:07.608339 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:23:07.608339 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:23:07.608339 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:23:07.608339 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:23:07.608339 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:23:07.608339 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:23:07.608339 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:23:07.608339 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:23:07.608339 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:23:07.609277 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:23:07.609277 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:23:07.609277 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:23:07.609277 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:23:07.609277 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:23:07.609277 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:23:07.609277 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:23:07.609277 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:23:07.609277 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:23:07.609277 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:23:07.609277 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:23:07.609277 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:23:07.609277 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:23:07.609277 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:23:07.609277 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:23:07.609277 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:23:07.610044 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:23:07.610044 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:23:07.610044 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:23:07.610044 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:23:07.610044 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:23:07.610044 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:23:07.610044 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:23:07.610044 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:23:07.610044 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:23:07.610044 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:23:07.610044 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:23:07.610044 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:23:07.613099 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 14/29] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Aug 24 05:23:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:23:07.614152 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41f85ce-cdf5-49ff-ae91-cdc5e326dba9 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:07.657978 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:07.679964 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c776779-eb0f-4fd1-b779-57b798484c75 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:07.711368 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:07.721900 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f55846f-d4d2-4a97-9027-f2e515173c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:07.723018 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f55846f-d4d2-4a97-9027-f2e515173c23 None admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:07.723631 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f55846f-d4d2-4a97-9027-f2e515173c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:07.724067 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f55846f-d4d2-4a97-9027-f2e515173c23 None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:07.725008 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f55846f-d4d2-4a97-9027-f2e515173c23 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:07.726565 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f55846f-d4d2-4a97-9027-f2e515173c23 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:07.744633 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41f85ce-cdf5-49ff-ae91-cdc5e326dba9 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:07.782165 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5efd5d2-1426-43b7-b020-51f3a43f31c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:07.782784 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efd5d2-1426-43b7-b020-51f3a43f31c9 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:07.783061 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efd5d2-1426-43b7-b020-51f3a43f31c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:07.783853 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efd5d2-1426-43b7-b020-51f3a43f31c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:07.784531 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5efd5d2-1426-43b7-b020-51f3a43f31c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:07.784833 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5efd5d2-1426-43b7-b020-51f3a43f31c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:07.786244 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5efd5d2-1426-43b7-b020-51f3a43f31c9 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:07.799851 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 15/30] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:23:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:07.805174 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdaf0d5-3877-4945-9b6b-008b9f2d0aaa None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:07.805174 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdaf0d5-3877-4945-9b6b-008b9f2d0aaa None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:07.805174 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdaf0d5-3877-4945-9b6b-008b9f2d0aaa None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:07.805174 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 16/31] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:07.805174 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdaf0d5-3877-4945-9b6b-008b9f2d0aaa None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:07.937039 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:07.945411 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdaf0d5-3877-4945-9b6b-008b9f2d0aaa None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:07.992531 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b736638-4fad-4800-aab2-6d8339242b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:07.992531 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b736638-4fad-4800-aab2-6d8339242b4d None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:07.992531 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b736638-4fad-4800-aab2-6d8339242b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:07.992531 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b736638-4fad-4800-aab2-6d8339242b4d None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:07.992982 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b736638-4fad-4800-aab2-6d8339242b4d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:07.992982 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b736638-4fad-4800-aab2-6d8339242b4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:07.992982 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b736638-4fad-4800-aab2-6d8339242b4d None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:07.993725 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 16/32] 10.222.0.15 () {60 vars in 1131 bytes} [Wed Aug 24 05:23:07 2022] POST /identity/v3/roles => generated 219 bytes in 456 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:23:07.995106 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f55846f-d4d2-4a97-9027-f2e515173c23 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:08.003629 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 17/33] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:23:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:08.009204 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292589c7-8005-4b4c-9d09-c237d453fee7 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:08.009204 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292589c7-8005-4b4c-9d09-c237d453fee7 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:08.009204 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292589c7-8005-4b4c-9d09-c237d453fee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:08.028504 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-292589c7-8005-4b4c-9d09-c237d453fee7 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:08.029988 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-292589c7-8005-4b4c-9d09-c237d453fee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:08.173179 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 18/34] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:08.174367 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292589c7-8005-4b4c-9d09-c237d453fee7 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:08.215474 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f55846f-d4d2-4a97-9027-f2e515173c23 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:08.266581 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044bfff8-0533-49b8-b509-7a5e22e1585d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:08.267394 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044bfff8-0533-49b8-b509-7a5e22e1585d None admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:08.268326 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044bfff8-0533-49b8-b509-7a5e22e1585d None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:08.269422 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044bfff8-0533-49b8-b509-7a5e22e1585d None admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:08.270637 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044bfff8-0533-49b8-b509-7a5e22e1585d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:08.273384 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044bfff8-0533-49b8-b509-7a5e22e1585d None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:08.310840 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:08.323394 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292589c7-8005-4b4c-9d09-c237d453fee7 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:08.360158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda2cb8b-b5e6-42c7-b83b-a49afdcecf7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:08.360937 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2cb8b-b5e6-42c7-b83b-a49afdcecf7e None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:08.361790 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2cb8b-b5e6-42c7-b83b-a49afdcecf7e None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:08.362142 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2cb8b-b5e6-42c7-b83b-a49afdcecf7e None admin] PATH_INFO: `/v3/users/359799b7d001453b90d984a05215cce6` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:08.363697 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda2cb8b-b5e6-42c7-b83b-a49afdcecf7e None admin] RBAC: Authorizing `identity:get_user(user_id=359799b7d001453b90d984a05215cce6)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:08.365240 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda2cb8b-b5e6-42c7-b83b-a49afdcecf7e None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:08.366841 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 19/35] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:23:08 2022] GET /identity/v3/users/359799b7d001453b90d984a05215cce6 => generated 247 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:08.368195 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d77511-c818-4033-a5b5-545080cfb660 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:08.368693 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d77511-c818-4033-a5b5-545080cfb660 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:08.368983 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d77511-c818-4033-a5b5-545080cfb660 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:08.370378 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 20/36] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:08.371317 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec91c4-8afe-4c2b-b5ca-bf5b17f472b3 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:08.371639 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec91c4-8afe-4c2b-b5ca-bf5b17f472b3 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:08.372197 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec91c4-8afe-4c2b-b5ca-bf5b17f472b3 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:08.373176 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 21/37] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:08.375643 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ec91c4-8afe-4c2b-b5ca-bf5b17f472b3 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:08.421489 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 17/38] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Aug 24 05:23:07 2022] POST /identity/v3/projects => generated 301 bytes in 427 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:23:08.422981 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b69e0-f30c-4bef-84f5-119ff4250c03 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:08.423406 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b69e0-f30c-4bef-84f5-119ff4250c03 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:08.423718 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b69e0-f30c-4bef-84f5-119ff4250c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:08.448264 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-1c6b69e0-f30c-4bef-84f5-119ff4250c03 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:08.451022 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-1c6b69e0-f30c-4bef-84f5-119ff4250c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:08.489804 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:08.509885 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ec91c4-8afe-4c2b-b5ca-bf5b17f472b3 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:08.544766 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 18/39] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:08.546184 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d81fbb-2f5a-4cf3-bb57-6014a7cf9b12 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:08.546473 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d81fbb-2f5a-4cf3-bb57-6014a7cf9b12 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:08.546753 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d81fbb-2f5a-4cf3-bb57-6014a7cf9b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:08.558187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e6750-45c7-43cd-ad60-acdfd36b15a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:08.558855 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e6750-45c7-43cd-ad60-acdfd36b15a5 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:08.559128 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e6750-45c7-43cd-ad60-acdfd36b15a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:08.559409 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e6750-45c7-43cd-ad60-acdfd36b15a5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:08.561573 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9e6750-45c7-43cd-ad60-acdfd36b15a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:08.563528 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9e6750-45c7-43cd-ad60-acdfd36b15a5 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:08.566262 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 22/40] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:23:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:23:08.567181 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e6750-45c7-43cd-ad60-acdfd36b15a5 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:08.573863 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-43d81fbb-2f5a-4cf3-bb57-6014a7cf9b12 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:08.575235 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-43d81fbb-2f5a-4cf3-bb57-6014a7cf9b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:08.601816 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:08.660891 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 19/41] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:08.661940 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d81fbb-2f5a-4cf3-bb57-6014a7cf9b12 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:08.712241 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e6750-45c7-43cd-ad60-acdfd36b15a5 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:08.756888 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca6561-7cc3-4227-991f-69d71059fdbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:08.757525 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca6561-7cc3-4227-991f-69d71059fdbc None admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:08.757919 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca6561-7cc3-4227-991f-69d71059fdbc None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:08.758204 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca6561-7cc3-4227-991f-69d71059fdbc None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:08.759017 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdca6561-7cc3-4227-991f-69d71059fdbc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:08.760497 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdca6561-7cc3-4227-991f-69d71059fdbc None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:08.879476 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:08.883339 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d81fbb-2f5a-4cf3-bb57-6014a7cf9b12 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:08.919892 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc210ee-9d86-4433-8355-87c02660ce0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:08.920605 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc210ee-9d86-4433-8355-87c02660ce0f None admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:08.920886 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc210ee-9d86-4433-8355-87c02660ce0f None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:08.921164 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc210ee-9d86-4433-8355-87c02660ce0f None admin] PATH_INFO: `/v3/domains/default/users/359799b7d001453b90d984a05215cce6/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:08.923130 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc210ee-9d86-4433-8355-87c02660ce0f None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=359799b7d001453b90d984a05215cce6, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:08.925463 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc210ee-9d86-4433-8355-87c02660ce0f None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:08.979242 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 23/42] 10.222.0.15 () {60 vars in 1131 bytes} [Wed Aug 24 05:23:08 2022] POST /identity/v3/roles => generated 225 bytes in 412 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:23:08.980336 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca6561-7cc3-4227-991f-69d71059fdbc None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:09.043328 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 20/43] 10.222.0.15 () {58 vars in 1361 bytes} [Wed Aug 24 05:23:08 2022] PUT /identity/v3/domains/default/users/359799b7d001453b90d984a05215cce6/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:23:09.053532 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc210ee-9d86-4433-8355-87c02660ce0f None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:09.106710 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:09.136681 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca6561-7cc3-4227-991f-69d71059fdbc None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:09.197301 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d874b3-3d7f-4257-9e1d-82bf24af0b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:09.197976 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d874b3-3d7f-4257-9e1d-82bf24af0b7f None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:09.198261 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d874b3-3d7f-4257-9e1d-82bf24af0b7f None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:09.199188 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d874b3-3d7f-4257-9e1d-82bf24af0b7f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:09.203204 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d874b3-3d7f-4257-9e1d-82bf24af0b7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:09.204878 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d874b3-3d7f-4257-9e1d-82bf24af0b7f None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:09.207282 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 24/44] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:23:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:09.213443 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d874b3-3d7f-4257-9e1d-82bf24af0b7f None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:09.242137 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc210ee-9d86-4433-8355-87c02660ce0f None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:09.295739 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41eab4-94c1-4fb8-88d6-485cd95999bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:09.296938 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41eab4-94c1-4fb8-88d6-485cd95999bf None admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:09.297443 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41eab4-94c1-4fb8-88d6-485cd95999bf None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:09.297914 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41eab4-94c1-4fb8-88d6-485cd95999bf None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:09.298613 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c41eab4-94c1-4fb8-88d6-485cd95999bf None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:09.301451 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c41eab4-94c1-4fb8-88d6-485cd95999bf None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:09.363409 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d874b3-3d7f-4257-9e1d-82bf24af0b7f None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:09.388644 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 21/45] 10.222.0.15 () {60 vars in 1131 bytes} [Wed Aug 24 05:23:09 2022] POST /identity/v3/roles => generated 223 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:23:09.409529 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff28533-fa75-47b1-8796-641548d3d5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:09.410211 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff28533-fa75-47b1-8796-641548d3d5f5 None admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:09.410522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff28533-fa75-47b1-8796-641548d3d5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:09.410802 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff28533-fa75-47b1-8796-641548d3d5f5 None admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:09.411482 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff28533-fa75-47b1-8796-641548d3d5f5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:09.413602 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff28533-fa75-47b1-8796-641548d3d5f5 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:09.513456 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 25/46] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Aug 24 05:23:09 2022] POST /identity/v3/projects => generated 312 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:23:11.135455 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae47c2-97c7-4fee-984d-8fa25b466cc4 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:11.135972 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae47c2-97c7-4fee-984d-8fa25b466cc4 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:11.136056 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae47c2-97c7-4fee-984d-8fa25b466cc4 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:11.137642 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 22/47] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:11.143607 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8992dc7-7048-419b-8bad-937439b4337a None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:11.144115 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8992dc7-7048-419b-8bad-937439b4337a None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:11.144286 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8992dc7-7048-419b-8bad-937439b4337a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:11.152073 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:11.273569 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-b8992dc7-7048-419b-8bad-937439b4337a None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:11.275140 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-b8992dc7-7048-419b-8bad-937439b4337a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:11.350157 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 26/48] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:11.572310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52614f3-1df7-4eaa-938a-ca5636711683 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:11.572581 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52614f3-1df7-4eaa-938a-ca5636711683 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:11.572877 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52614f3-1df7-4eaa-938a-ca5636711683 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:11.574226 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 23/49] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:11.579685 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115b062-fecd-4cd4-ad93-2acab3f68f53 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:11.580117 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115b062-fecd-4cd4-ad93-2acab3f68f53 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:11.580456 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115b062-fecd-4cd4-ad93-2acab3f68f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:11.599503 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-d115b062-fecd-4cd4-ad93-2acab3f68f53 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:11.600781 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-d115b062-fecd-4cd4-ad93-2acab3f68f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:11.625766 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52614f3-1df7-4eaa-938a-ca5636711683 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:11.645873 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:11.682672 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 27/50] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:11.743441 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52614f3-1df7-4eaa-938a-ca5636711683 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:11.785175 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bfeab0-45f7-43ff-ae73-d43d2faa6a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:11.785824 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bfeab0-45f7-43ff-ae73-d43d2faa6a94 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:11.786109 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bfeab0-45f7-43ff-ae73-d43d2faa6a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:11.786395 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bfeab0-45f7-43ff-ae73-d43d2faa6a94 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:11.797719 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bfeab0-45f7-43ff-ae73-d43d2faa6a94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:23:11.803434 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-74bfeab0-45f7-43ff-ae73-d43d2faa6a94 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:23:11.803434 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:23:11.803434 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:23:11.803434 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:23:11.803434 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:23:11.803434 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:23:11.803434 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:23:11.803434 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:23:11.803434 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:23:11.803434 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:23:11.803434 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:23:11.803434 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:23:11.803434 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:23:11.803434 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:23:11.803434 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:23:11.803434 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:23:11.804310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:23:11.804310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:23:11.804310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:23:11.804310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:23:11.804310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:23:11.804310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:23:11.804310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:23:11.804310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:23:11.804310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:23:11.804310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:23:11.804310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:23:11.804310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:23:11.804310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:23:11.804310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:23:11.804310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:23:11.804310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:23:11.805358 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:23:11.805358 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:23:11.805358 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:23:11.805358 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:23:11.805358 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:23:11.805358 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:23:11.805358 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:23:11.805358 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:23:11.805358 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:23:11.805358 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:23:11.805358 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:23:11.805358 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:23:11.805358 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 24/51] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Aug 24 05:23:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:23:11.810436 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d115b062-fecd-4cd4-ad93-2acab3f68f53 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:11.879131 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:11.921511 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d115b062-fecd-4cd4-ad93-2acab3f68f53 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:11.956246 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bfeab0-45f7-43ff-ae73-d43d2faa6a94 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:11.960877 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c5ac14-b3d8-4908-9429-03f568c2ae32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:11.961560 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5ac14-b3d8-4908-9429-03f568c2ae32 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:11.961844 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5ac14-b3d8-4908-9429-03f568c2ae32 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:11.962121 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5ac14-b3d8-4908-9429-03f568c2ae32 None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:11.962808 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c5ac14-b3d8-4908-9429-03f568c2ae32 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:11.963154 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c5ac14-b3d8-4908-9429-03f568c2ae32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:11.964786 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c5ac14-b3d8-4908-9429-03f568c2ae32 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:11.973955 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 28/52] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:23:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:11.979881 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c5ac14-b3d8-4908-9429-03f568c2ae32 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:12.045230 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:12.067794 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bfeab0-45f7-43ff-ae73-d43d2faa6a94 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:12.091557 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c5ac14-b3d8-4908-9429-03f568c2ae32 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:12.105372 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e158c672-eec5-4e52-ad9e-089bfad9b322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:12.105918 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e158c672-eec5-4e52-ad9e-089bfad9b322 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:12.106203 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e158c672-eec5-4e52-ad9e-089bfad9b322 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:12.106672 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e158c672-eec5-4e52-ad9e-089bfad9b322 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:12.108346 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e158c672-eec5-4e52-ad9e-089bfad9b322 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:12.110212 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e158c672-eec5-4e52-ad9e-089bfad9b322 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:12.111922 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 25/53] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:23:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:23:12.117870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e158c672-eec5-4e52-ad9e-089bfad9b322 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:12.124850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:12.136223 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc1cb67-1c64-45a6-98db-cdc3d6554e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:12.136223 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc1cb67-1c64-45a6-98db-cdc3d6554e7c None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:12.136223 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc1cb67-1c64-45a6-98db-cdc3d6554e7c None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:12.136223 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc1cb67-1c64-45a6-98db-cdc3d6554e7c None admin] PATH_INFO: `/v3/users/359799b7d001453b90d984a05215cce6` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:12.136869 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc1cb67-1c64-45a6-98db-cdc3d6554e7c None admin] RBAC: Authorizing `identity:get_user(user_id=359799b7d001453b90d984a05215cce6)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:12.138357 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc1cb67-1c64-45a6-98db-cdc3d6554e7c None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:12.139974 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 29/54] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:23:11 2022] GET /identity/v3/users/359799b7d001453b90d984a05215cce6 => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:23:12.146544 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc1cb67-1c64-45a6-98db-cdc3d6554e7c None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:12.237864 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e158c672-eec5-4e52-ad9e-089bfad9b322 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:12.273686 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc1cb67-1c64-45a6-98db-cdc3d6554e7c None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:12.277951 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f56ad27-1eb4-4b2d-a25e-411a0bc8b2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:12.278579 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f56ad27-1eb4-4b2d-a25e-411a0bc8b2ae None admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:12.278863 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f56ad27-1eb4-4b2d-a25e-411a0bc8b2ae None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:12.279146 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f56ad27-1eb4-4b2d-a25e-411a0bc8b2ae None admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:12.279814 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f56ad27-1eb4-4b2d-a25e-411a0bc8b2ae None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:12.281617 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f56ad27-1eb4-4b2d-a25e-411a0bc8b2ae None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:12.317009 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a004ae49-695d-4045-b893-488a9a5e0ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:12.319557 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a004ae49-695d-4045-b893-488a9a5e0ab1 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:12.319557 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a004ae49-695d-4045-b893-488a9a5e0ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:12.319557 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a004ae49-695d-4045-b893-488a9a5e0ab1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:12.319557 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a004ae49-695d-4045-b893-488a9a5e0ab1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:12.320842 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a004ae49-695d-4045-b893-488a9a5e0ab1 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:12.322433 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 30/55] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:23:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:23:12.328776 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a004ae49-695d-4045-b893-488a9a5e0ab1 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:12.379634 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:12.386689 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 26/56] 10.222.0.15 () {60 vars in 1140 bytes} [Wed Aug 24 05:23:12 2022] POST /identity/v3/projects => generated 298 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:23:12.395730 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:12.441985 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a004ae49-695d-4045-b893-488a9a5e0ab1 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:12.480129 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c882bc07-7381-4a63-a6d3-cb522cec0352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:12.480803 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c882bc07-7381-4a63-a6d3-cb522cec0352 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:12.481086 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c882bc07-7381-4a63-a6d3-cb522cec0352 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:12.481364 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c882bc07-7381-4a63-a6d3-cb522cec0352 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:12.482134 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c882bc07-7381-4a63-a6d3-cb522cec0352 None admin] RBAC: Adding query filter params (role.id=e013ad4d436f4cb0a9bf7debb7d13dc7, scope.domain.id=default, user.id=359799b7d001453b90d984a05215cce6) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:12.482448 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c882bc07-7381-4a63-a6d3-cb522cec0352 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:12.483993 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c882bc07-7381-4a63-a6d3-cb522cec0352 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:12.494501 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 31/57] 10.222.0.15 () {56 vars in 1322 bytes} [Wed Aug 24 05:23:12 2022] GET /identity/v3/role_assignments?user.id=359799b7d001453b90d984a05215cce6&scope.domain.id=default&role.id=e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 530 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:23:14.479770 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e25a4-7fd4-4323-83e9-27d18d51a89e None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:14.480275 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e25a4-7fd4-4323-83e9-27d18d51a89e None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:14.480275 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e25a4-7fd4-4323-83e9-27d18d51a89e None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:14.481670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 27/58] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:14.487097 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d938c5-12e0-4ca1-8340-f2c9f7a2e1d7 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:14.487510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d938c5-12e0-4ca1-8340-f2c9f7a2e1d7 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:14.487660 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d938c5-12e0-4ca1-8340-f2c9f7a2e1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:14.506055 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-04d938c5-12e0-4ca1-8340-f2c9f7a2e1d7 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:14.507240 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-04d938c5-12e0-4ca1-8340-f2c9f7a2e1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:14.579260 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 32/59] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:14.895526 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518e25a4-7fd4-4323-83e9-27d18d51a89e None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:15.053401 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518e25a4-7fd4-4323-83e9-27d18d51a89e None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:15.090010 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5357ccc-3446-479b-954a-3a6a6e13c54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:15.090649 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5357ccc-3446-479b-954a-3a6a6e13c54c None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:15.090932 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5357ccc-3446-479b-954a-3a6a6e13c54c None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:15.091210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5357ccc-3446-479b-954a-3a6a6e13c54c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:15.092390 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5357ccc-3446-479b-954a-3a6a6e13c54c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:15.093897 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5357ccc-3446-479b-954a-3a6a6e13c54c None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:15.096077 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 28/60] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:23:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:15.101696 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d938c5-12e0-4ca1-8340-f2c9f7a2e1d7 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:15.164982 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:15.239317 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d938c5-12e0-4ca1-8340-f2c9f7a2e1d7 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:15.282776 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b445690e-55b1-4ad2-99f7-d6416b9b4078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:15.283418 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445690e-55b1-4ad2-99f7-d6416b9b4078 None admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:15.283959 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445690e-55b1-4ad2-99f7-d6416b9b4078 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:15.284073 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445690e-55b1-4ad2-99f7-d6416b9b4078 None admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:15.284901 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b445690e-55b1-4ad2-99f7-d6416b9b4078 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:15.286392 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b445690e-55b1-4ad2-99f7-d6416b9b4078 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:15.415498 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 33/61] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Aug 24 05:23:15 2022] POST /identity/v3/users => generated 275 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:23:18.543430 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b2648-e197-4745-90c5-31705c321b8b None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:18.545213 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b2648-e197-4745-90c5-31705c321b8b None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:18.546093 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b2648-e197-4745-90c5-31705c321b8b None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:18.548835 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 29/62] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:18.554820 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317fad8-345f-47fb-a794-f3969c89ded4 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:18.555870 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317fad8-345f-47fb-a794-f3969c89ded4 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:18.556367 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317fad8-345f-47fb-a794-f3969c89ded4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:18.585551 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-2317fad8-345f-47fb-a794-f3969c89ded4 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:18.587174 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-2317fad8-345f-47fb-a794-f3969c89ded4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:18.662602 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceedf06-4f08-4c4e-8349-7533754ab668 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:18.663252 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceedf06-4f08-4c4e-8349-7533754ab668 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:18.663734 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceedf06-4f08-4c4e-8349-7533754ab668 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:18.665369 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 30/63] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:23:18.671492 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc8d76-d572-4e54-8b2d-51d78087e75f None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:18.671783 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc8d76-d572-4e54-8b2d-51d78087e75f None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:18.672069 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc8d76-d572-4e54-8b2d-51d78087e75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:18.689988 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 34/64] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:18.706418 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-2abc8d76-d572-4e54-8b2d-51d78087e75f None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:18.708193 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-2abc8d76-d572-4e54-8b2d-51d78087e75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:18.803691 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:18.812307 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 31/65] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:23:18.841042 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453ac47-31cd-4036-a975-021d7002c16a None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:18.841578 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453ac47-31cd-4036-a975-021d7002c16a None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:18.842027 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453ac47-31cd-4036-a975-021d7002c16a None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:18.843545 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 35/66] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:18.849976 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322588b7-ae4c-47a4-a302-24d909c55cd9 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:18.850482 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322588b7-ae4c-47a4-a302-24d909c55cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:18.850915 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322588b7-ae4c-47a4-a302-24d909c55cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:18.881600 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-322588b7-ae4c-47a4-a302-24d909c55cd9 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:18.883072 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-322588b7-ae4c-47a4-a302-24d909c55cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:18.974410 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 32/67] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:19.120355 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b453ac47-31cd-4036-a975-021d7002c16a None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:19.173489 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322588b7-ae4c-47a4-a302-24d909c55cd9 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:19.192772 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:19.273123 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b453ac47-31cd-4036-a975-021d7002c16a None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:19.300371 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322588b7-ae4c-47a4-a302-24d909c55cd9 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:19.319602 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d3685-8711-4838-9e48-4799da983815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:19.320257 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d3685-8711-4838-9e48-4799da983815 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:19.320727 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d3685-8711-4838-9e48-4799da983815 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:19.321013 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d3685-8711-4838-9e48-4799da983815 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:19.331539 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:19.335812 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195d3685-8711-4838-9e48-4799da983815 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:23:19.341497 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d8b03-4674-4949-a31e-f349b9de3c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:19.343590 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-195d3685-8711-4838-9e48-4799da983815 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:23:19.343590 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:23:19.343590 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:23:19.343590 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:23:19.343590 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:23:19.343590 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:23:19.343590 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:23:19.343590 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:23:19.343590 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:23:19.343590 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:23:19.343590 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:23:19.343590 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:23:19.343590 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:23:19.343590 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:23:19.343590 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:23:19.343590 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:23:19.344518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:23:19.344518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:23:19.344518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:23:19.344518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:23:19.344518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:23:19.344518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:23:19.344518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:23:19.344518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:23:19.344518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:23:19.344518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:23:19.344518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:23:19.344518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:23:19.344518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:23:19.344518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:23:19.344518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:23:19.344518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:23:19.345943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:23:19.345943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:23:19.345943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:23:19.345943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:23:19.345943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:23:19.345943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:23:19.345943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:23:19.345943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:23:19.345943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:23:19.345943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:23:19.345943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:23:19.345943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:23:19.346742 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d8b03-4674-4949-a31e-f349b9de3c47 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:19.346742 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d8b03-4674-4949-a31e-f349b9de3c47 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:19.346742 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d8b03-4674-4949-a31e-f349b9de3c47 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:19.346938 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 36/68] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:23:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:23:19.346938 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3c8c2-28e1-4e74-af24-95d99a580df3 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:19.346938 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3c8c2-28e1-4e74-af24-95d99a580df3 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:19.346938 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3c8c2-28e1-4e74-af24-95d99a580df3 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:19.346938 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 37/69] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:19.347892 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3c8c2-28e1-4e74-af24-95d99a580df3 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:19.360037 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d8b03-4674-4949-a31e-f349b9de3c47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:23:19.367121 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-b30d8b03-4674-4949-a31e-f349b9de3c47 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:23:19.367121 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:23:19.367121 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:23:19.367121 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:23:19.367121 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:23:19.367121 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:23:19.367121 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:23:19.367121 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:23:19.367121 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:23:19.367121 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:23:19.367121 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:23:19.367121 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:23:19.367121 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:23:19.367121 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:23:19.367121 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:23:19.367121 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:23:19.368089 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:23:19.368089 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:23:19.368089 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:23:19.368089 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:23:19.368089 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:23:19.368089 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:23:19.368089 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:23:19.368089 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:23:19.368089 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:23:19.368089 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:23:19.368089 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:23:19.368089 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:23:19.368089 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:23:19.368089 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:23:19.368089 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:23:19.368089 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:23:19.368881 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:23:19.368881 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:23:19.368881 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:23:19.368881 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:23:19.368881 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:23:19.368881 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:23:19.368881 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:23:19.368881 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:23:19.368881 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:23:19.368881 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:23:19.368881 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:23:19.368881 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:23:19.376890 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 33/70] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Aug 24 05:23:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:23:19.376890 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d8b03-4674-4949-a31e-f349b9de3c47 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:19.468417 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3c8c2-28e1-4e74-af24-95d99a580df3 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:19.470965 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:19.502195 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d8b03-4674-4949-a31e-f349b9de3c47 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:19.507988 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff35e4a-1fd5-42bc-b96a-72c2d40fa099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:19.508674 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff35e4a-1fd5-42bc-b96a-72c2d40fa099 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:19.508964 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff35e4a-1fd5-42bc-b96a-72c2d40fa099 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:19.509250 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff35e4a-1fd5-42bc-b96a-72c2d40fa099 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:19.520940 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff35e4a-1fd5-42bc-b96a-72c2d40fa099 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:23:19.526179 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-4ff35e4a-1fd5-42bc-b96a-72c2d40fa099 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:23:19.526179 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:23:19.526179 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:23:19.526179 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:23:19.526179 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:23:19.526179 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:23:19.526179 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:23:19.526179 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:23:19.526179 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:23:19.526179 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:23:19.526179 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:23:19.526179 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:23:19.526179 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:23:19.526179 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:23:19.526179 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:23:19.526179 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:23:19.527057 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:23:19.527057 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:23:19.527057 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:23:19.527057 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:23:19.527057 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:23:19.527057 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:23:19.527057 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:23:19.527057 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:23:19.527057 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:23:19.527057 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:23:19.527057 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:23:19.527057 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:23:19.527057 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:23:19.527057 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:23:19.527057 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:23:19.527057 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:23:19.533497 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:23:19.533497 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:23:19.533497 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:23:19.533497 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:23:19.533497 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:23:19.533497 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:23:19.533497 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:23:19.533497 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:23:19.533497 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:23:19.533497 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:23:19.533497 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:23:19.533497 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:23:19.533497 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 38/71] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Aug 24 05:23:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:23:19.533497 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d0e8c-5435-4d84-966b-3f4e845b7183 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:19.534269 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d0e8c-5435-4d84-966b-3f4e845b7183 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:19.534269 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d0e8c-5435-4d84-966b-3f4e845b7183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:19.538317 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d6c99e-0933-4eff-a20a-dd1b415b16f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:19.538818 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6c99e-0933-4eff-a20a-dd1b415b16f2 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:19.539818 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6c99e-0933-4eff-a20a-dd1b415b16f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:19.539818 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6c99e-0933-4eff-a20a-dd1b415b16f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:19.540055 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d6c99e-0933-4eff-a20a-dd1b415b16f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:19.540334 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d6c99e-0933-4eff-a20a-dd1b415b16f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:19.541779 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d6c99e-0933-4eff-a20a-dd1b415b16f2 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:19.551495 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-229d0e8c-5435-4d84-966b-3f4e845b7183 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:19.553050 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-229d0e8c-5435-4d84-966b-3f4e845b7183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:19.553222 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 34/72] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Aug 24 05:23:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:19.553699 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d6c99e-0933-4eff-a20a-dd1b415b16f2 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:19.608923 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:19.630802 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 39/73] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:19.633530 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99da0fb-feb3-4376-8721-58847a04a415 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:19.633828 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99da0fb-feb3-4376-8721-58847a04a415 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:19.634120 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99da0fb-feb3-4376-8721-58847a04a415 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:19.636691 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 40/74] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:19.636691 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99da0fb-feb3-4376-8721-58847a04a415 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:19.670053 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d6c99e-0933-4eff-a20a-dd1b415b16f2 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:19.716351 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8e854e-7fe7-4d6b-b86c-4ce761bc21fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:19.717000 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e854e-7fe7-4d6b-b86c-4ce761bc21fa None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:19.717267 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e854e-7fe7-4d6b-b86c-4ce761bc21fa None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:19.717822 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e854e-7fe7-4d6b-b86c-4ce761bc21fa None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:19.718520 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8e854e-7fe7-4d6b-b86c-4ce761bc21fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:19.719429 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8e854e-7fe7-4d6b-b86c-4ce761bc21fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:19.721205 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8e854e-7fe7-4d6b-b86c-4ce761bc21fa None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:19.731611 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 35/75] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:23:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:19.736932 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8e854e-7fe7-4d6b-b86c-4ce761bc21fa None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:19.758822 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:19.770617 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99da0fb-feb3-4376-8721-58847a04a415 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:19.812472 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255fd797-5742-43ae-97dd-48734dc94571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:19.813144 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255fd797-5742-43ae-97dd-48734dc94571 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:19.813482 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255fd797-5742-43ae-97dd-48734dc94571 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:19.813779 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255fd797-5742-43ae-97dd-48734dc94571 None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:19.814510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255fd797-5742-43ae-97dd-48734dc94571 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:19.814795 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255fd797-5742-43ae-97dd-48734dc94571 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:19.816289 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255fd797-5742-43ae-97dd-48734dc94571 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:19.825315 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 41/76] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:23:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:19.831239 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1980ea-223a-464b-a216-a7ff67d60b8a None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:19.831533 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1980ea-223a-464b-a216-a7ff67d60b8a None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:19.831816 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1980ea-223a-464b-a216-a7ff67d60b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:19.852046 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-5f1980ea-223a-464b-a216-a7ff67d60b8a None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:19.853347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-5f1980ea-223a-464b-a216-a7ff67d60b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:19.864426 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8e854e-7fe7-4d6b-b86c-4ce761bc21fa None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:19.875117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:19.902221 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b04d5a5-a2fe-4f7d-b002-9a5ae22f1f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:19.902706 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04d5a5-a2fe-4f7d-b002-9a5ae22f1f68 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:19.902993 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04d5a5-a2fe-4f7d-b002-9a5ae22f1f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:19.903654 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04d5a5-a2fe-4f7d-b002-9a5ae22f1f68 None admin] PATH_INFO: `/v3/users/ab58de7f803c4cd29f61931966b168d5` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:19.919423 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b04d5a5-a2fe-4f7d-b002-9a5ae22f1f68 None admin] RBAC: Authorizing `identity:get_user(user_id=ab58de7f803c4cd29f61931966b168d5)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:19.920968 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b04d5a5-a2fe-4f7d-b002-9a5ae22f1f68 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:19.922679 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 36/77] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:23:19 2022] GET /identity/v3/users/ab58de7f803c4cd29f61931966b168d5 => generated 275 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:19.925364 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b04d5a5-a2fe-4f7d-b002-9a5ae22f1f68 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:19.930718 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 42/78] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:19.934877 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1980ea-223a-464b-a216-a7ff67d60b8a None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:20.044617 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:20.053164 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b04d5a5-a2fe-4f7d-b002-9a5ae22f1f68 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:20.062924 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1980ea-223a-464b-a216-a7ff67d60b8a None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:20.092648 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d9a51-2add-4cd8-b3b9-4ace8be8f380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:20.093359 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d9a51-2add-4cd8-b3b9-4ace8be8f380 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:20.093646 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d9a51-2add-4cd8-b3b9-4ace8be8f380 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:20.093927 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d9a51-2add-4cd8-b3b9-4ace8be8f380 None admin] PATH_INFO: `/v3/users/359799b7d001453b90d984a05215cce6` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:20.095632 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d9a51-2add-4cd8-b3b9-4ace8be8f380 None admin] RBAC: Authorizing `identity:get_user(user_id=359799b7d001453b90d984a05215cce6)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:20.097314 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d9a51-2add-4cd8-b3b9-4ace8be8f380 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:20.099057 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 37/79] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:23:19 2022] GET /identity/v3/users/359799b7d001453b90d984a05215cce6 => generated 247 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:20.099575 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d9a51-2add-4cd8-b3b9-4ace8be8f380 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:20.117268 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d85dbc-aad6-498c-ac29-1627495f75a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:20.118015 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d85dbc-aad6-498c-ac29-1627495f75a2 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:20.118375 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d85dbc-aad6-498c-ac29-1627495f75a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:20.118731 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d85dbc-aad6-498c-ac29-1627495f75a2 None admin] PATH_INFO: `/v3/users/ab58de7f803c4cd29f61931966b168d5` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:20.119996 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d85dbc-aad6-498c-ac29-1627495f75a2 None admin] RBAC: Authorizing `identity:get_user(user_id=ab58de7f803c4cd29f61931966b168d5)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:20.121518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d85dbc-aad6-498c-ac29-1627495f75a2 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:20.128112 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 43/80] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:23:19 2022] GET /identity/v3/users/ab58de7f803c4cd29f61931966b168d5 => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:20.128112 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d85dbc-aad6-498c-ac29-1627495f75a2 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:20.141829 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:20.230548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d9a51-2add-4cd8-b3b9-4ace8be8f380 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:20.246670 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d85dbc-aad6-498c-ac29-1627495f75a2 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:20.265958 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04634aa8-9b92-4956-8eb4-6ca3de305b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:20.266583 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04634aa8-9b92-4956-8eb4-6ca3de305b62 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:20.266866 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04634aa8-9b92-4956-8eb4-6ca3de305b62 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:20.267303 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04634aa8-9b92-4956-8eb4-6ca3de305b62 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:20.277690 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04634aa8-9b92-4956-8eb4-6ca3de305b62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:23:20.281424 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0492666-19f5-4cac-ae62-2fed8eca0be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:20.282114 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0492666-19f5-4cac-ae62-2fed8eca0be9 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:20.282408 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0492666-19f5-4cac-ae62-2fed8eca0be9 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:20.282693 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0492666-19f5-4cac-ae62-2fed8eca0be9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:20.282793 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-04634aa8-9b92-4956-8eb4-6ca3de305b62 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:23:20.282793 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:23:20.282793 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:23:20.282793 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:23:20.282793 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:23:20.282793 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:23:20.282793 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:23:20.282793 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:23:20.282793 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:23:20.282793 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:23:20.282793 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:23:20.282793 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:23:20.282793 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:23:20.282793 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:23:20.282793 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:23:20.282793 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:23:20.283646 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:23:20.283646 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:23:20.283646 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:23:20.283646 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:23:20.283646 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:23:20.283646 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:23:20.283646 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:23:20.283646 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:23:20.283646 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:23:20.283646 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:23:20.283646 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:23:20.283646 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:23:20.283646 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:23:20.283646 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:23:20.283646 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:23:20.283646 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:23:20.284519 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:23:20.284519 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:23:20.284519 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:23:20.284519 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:23:20.284519 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:23:20.284519 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:23:20.284519 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:23:20.284519 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:23:20.284519 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:23:20.284519 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:23:20.284519 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:23:20.284519 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:23:20.284519 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 38/81] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:23:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:23:20.286774 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04634aa8-9b92-4956-8eb4-6ca3de305b62 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:20.319945 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:20.386921 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0492666-19f5-4cac-ae62-2fed8eca0be9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:20.388448 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0492666-19f5-4cac-ae62-2fed8eca0be9 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:20.389884 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:20.396597 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04634aa8-9b92-4956-8eb4-6ca3de305b62 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:20.433988 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19261fba-e068-4ea0-ad98-ffca0cc9f0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:20.434592 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19261fba-e068-4ea0-ad98-ffca0cc9f0ec None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:20.434872 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19261fba-e068-4ea0-ad98-ffca0cc9f0ec None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:20.435149 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19261fba-e068-4ea0-ad98-ffca0cc9f0ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:20.512098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19261fba-e068-4ea0-ad98-ffca0cc9f0ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:20.513522 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19261fba-e068-4ea0-ad98-ffca0cc9f0ec None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:20.522465 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 44/82] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:23:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:23:20.523343 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0492666-19f5-4cac-ae62-2fed8eca0be9 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:20.582586 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:20.635081 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:20.643115 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 39/83] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:23:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:23:20.643801 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19261fba-e068-4ea0-ad98-ffca0cc9f0ec None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:20.646509 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0492666-19f5-4cac-ae62-2fed8eca0be9 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:20.687811 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3166284f-d084-4c75-87e6-48a1b4ca0fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:20.688787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3166284f-d084-4c75-87e6-48a1b4ca0fa8 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:20.689388 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3166284f-d084-4c75-87e6-48a1b4ca0fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:20.689889 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3166284f-d084-4c75-87e6-48a1b4ca0fa8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:20.771986 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3166284f-d084-4c75-87e6-48a1b4ca0fa8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:20.773392 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3166284f-d084-4c75-87e6-48a1b4ca0fa8 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:20.774629 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19261fba-e068-4ea0-ad98-ffca0cc9f0ec None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:20.813766 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e216df3-ebb4-4d9f-9f19-8b9ecf00baa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:20.815137 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e216df3-ebb4-4d9f-9f19-8b9ecf00baa8 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:20.815425 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e216df3-ebb4-4d9f-9f19-8b9ecf00baa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:20.815699 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e216df3-ebb4-4d9f-9f19-8b9ecf00baa8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:20.816930 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e216df3-ebb4-4d9f-9f19-8b9ecf00baa8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:20.818404 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e216df3-ebb4-4d9f-9f19-8b9ecf00baa8 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:20.820054 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 40/84] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:23:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:20.820054 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e216df3-ebb4-4d9f-9f19-8b9ecf00baa8 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:20.836728 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:20.886947 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 45/85] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:23:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:23:20.888038 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3166284f-d084-4c75-87e6-48a1b4ca0fa8 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:20.893059 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:20.941638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e216df3-ebb4-4d9f-9f19-8b9ecf00baa8 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:20.980265 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e515f3-e8b0-4786-8e47-773d73486514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:20.981046 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e515f3-e8b0-4786-8e47-773d73486514 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:20.981419 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e515f3-e8b0-4786-8e47-773d73486514 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:20.981732 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e515f3-e8b0-4786-8e47-773d73486514 None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:20.982390 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e515f3-e8b0-4786-8e47-773d73486514 None admin] RBAC: Adding query filter params (name=member) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:20.982814 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e515f3-e8b0-4786-8e47-773d73486514 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:20.984378 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e515f3-e8b0-4786-8e47-773d73486514 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:20.995776 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3166284f-d084-4c75-87e6-48a1b4ca0fa8 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:20.996777 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 41/86] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Aug 24 05:23:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:20.997789 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e515f3-e8b0-4786-8e47-773d73486514 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:21.050701 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa74ac-6ef0-436e-af89-9ea478017c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:21.051292 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa74ac-6ef0-436e-af89-9ea478017c75 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:21.051582 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa74ac-6ef0-436e-af89-9ea478017c75 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:21.051896 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa74ac-6ef0-436e-af89-9ea478017c75 None admin] PATH_INFO: `/v3/projects/4c7185cb4791495281d3a16293793beb` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:21.054346 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fa74ac-6ef0-436e-af89-9ea478017c75 None admin] RBAC: Authorizing `identity:get_project(project_id=4c7185cb4791495281d3a16293793beb)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:21.055877 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fa74ac-6ef0-436e-af89-9ea478017c75 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:21.057796 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 46/87] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:23:20 2022] GET /identity/v3/projects/4c7185cb4791495281d3a16293793beb => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:21.059396 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa74ac-6ef0-436e-af89-9ea478017c75 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:21.083485 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:21.139675 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e515f3-e8b0-4786-8e47-773d73486514 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:21.169257 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa74ac-6ef0-436e-af89-9ea478017c75 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:21.172974 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:21.186581 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae96e2-baad-4eab-928b-81f4e73c67e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:21.187996 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae96e2-baad-4eab-928b-81f4e73c67e7 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:21.187996 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae96e2-baad-4eab-928b-81f4e73c67e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:21.187996 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae96e2-baad-4eab-928b-81f4e73c67e7 None admin] PATH_INFO: `/v3/projects/4c7185cb4791495281d3a16293793beb` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:21.189228 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ae96e2-baad-4eab-928b-81f4e73c67e7 None admin] RBAC: Authorizing `identity:get_project(project_id=4c7185cb4791495281d3a16293793beb)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:21.190692 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ae96e2-baad-4eab-928b-81f4e73c67e7 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:21.192461 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 42/88] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:23:20 2022] GET /identity/v3/projects/4c7185cb4791495281d3a16293793beb => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:21.193359 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae96e2-baad-4eab-928b-81f4e73c67e7 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:21.210356 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7a2cb7-7033-47e1-8df3-221853f921f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:21.210979 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a2cb7-7033-47e1-8df3-221853f921f6 None admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:21.211268 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a2cb7-7033-47e1-8df3-221853f921f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:21.211553 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a2cb7-7033-47e1-8df3-221853f921f6 None admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:21.212210 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7a2cb7-7033-47e1-8df3-221853f921f6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:21.213727 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7a2cb7-7033-47e1-8df3-221853f921f6 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:21.299179 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae96e2-baad-4eab-928b-81f4e73c67e7 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:21.310965 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 47/89] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Aug 24 05:23:21 2022] POST /identity/v3/users => generated 289 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:23:21.312155 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7a2cb7-7033-47e1-8df3-221853f921f6 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:21.341703 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55902cc-bd80-4744-bfcd-618611cadfa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:21.342359 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55902cc-bd80-4744-bfcd-618611cadfa2 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:21.342824 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55902cc-bd80-4744-bfcd-618611cadfa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:21.342943 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55902cc-bd80-4744-bfcd-618611cadfa2 None admin] PATH_INFO: `/v3/projects/4c7185cb4791495281d3a16293793beb` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:21.344612 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55902cc-bd80-4744-bfcd-618611cadfa2 None admin] RBAC: Authorizing `identity:get_project(project_id=4c7185cb4791495281d3a16293793beb)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:21.346771 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55902cc-bd80-4744-bfcd-618611cadfa2 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:21.348795 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 43/90] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:23:21 2022] GET /identity/v3/projects/4c7185cb4791495281d3a16293793beb => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:21.349503 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55902cc-bd80-4744-bfcd-618611cadfa2 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:21.424639 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:21.431554 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:21.454089 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7a2cb7-7033-47e1-8df3-221853f921f6 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:21.481555 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55902cc-bd80-4744-bfcd-618611cadfa2 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:21.508900 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b03443-1450-41a2-9168-5431be27650d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:21.510102 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b03443-1450-41a2-9168-5431be27650d None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:21.510102 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b03443-1450-41a2-9168-5431be27650d None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:21.510102 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b03443-1450-41a2-9168-5431be27650d None admin] PATH_INFO: `/v3/users/ab58de7f803c4cd29f61931966b168d5` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:21.516728 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b03443-1450-41a2-9168-5431be27650d None admin] RBAC: Authorizing `identity:get_user(user_id=ab58de7f803c4cd29f61931966b168d5)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:21.516728 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b03443-1450-41a2-9168-5431be27650d None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:21.516728 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 48/91] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:23:21 2022] GET /identity/v3/users/ab58de7f803c4cd29f61931966b168d5 => generated 275 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:21.516728 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b03443-1450-41a2-9168-5431be27650d None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:21.524126 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce316e2-2a46-4479-b708-70844207090a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:21.524776 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce316e2-2a46-4479-b708-70844207090a None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:21.525073 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce316e2-2a46-4479-b708-70844207090a None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:21.525345 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce316e2-2a46-4479-b708-70844207090a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:21.526094 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce316e2-2a46-4479-b708-70844207090a None admin] RBAC: Adding query filter params (role.id=87862cd6eaeb4d5fae089a801aac4f87, scope.project.id=4c7185cb4791495281d3a16293793beb, user.id=ab58de7f803c4cd29f61931966b168d5) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:21.526411 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce316e2-2a46-4479-b708-70844207090a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:21.527918 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce316e2-2a46-4479-b708-70844207090a None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:21.535079 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 44/92] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:23:21 2022] GET /identity/v3/role_assignments?user.id=ab58de7f803c4cd29f61931966b168d5&scope.project.id=4c7185cb4791495281d3a16293793beb&role.id=87862cd6eaeb4d5fae089a801aac4f87 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:21.535922 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce316e2-2a46-4479-b708-70844207090a None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:21.674497 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b03443-1450-41a2-9168-5431be27650d None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:21.696760 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce316e2-2a46-4479-b708-70844207090a None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:21.726017 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666fb2d8-6c4a-4abc-9a42-bcf1971a89e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:21.726913 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666fb2d8-6c4a-4abc-9a42-bcf1971a89e2 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:21.727199 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666fb2d8-6c4a-4abc-9a42-bcf1971a89e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:21.727481 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666fb2d8-6c4a-4abc-9a42-bcf1971a89e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:21.728221 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666fb2d8-6c4a-4abc-9a42-bcf1971a89e2 None admin] RBAC: Adding query filter params (role.id=e013ad4d436f4cb0a9bf7debb7d13dc7, scope.project.id=4c7185cb4791495281d3a16293793beb, user.id=359799b7d001453b90d984a05215cce6) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:21.728566 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666fb2d8-6c4a-4abc-9a42-bcf1971a89e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:21.730040 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666fb2d8-6c4a-4abc-9a42-bcf1971a89e2 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:21.730655 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:21.737477 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 49/93] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:23:21 2022] GET /identity/v3/role_assignments?user.id=359799b7d001453b90d984a05215cce6&scope.project.id=4c7185cb4791495281d3a16293793beb&role.id=e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:21.739026 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666fb2d8-6c4a-4abc-9a42-bcf1971a89e2 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:21.752792 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ac4b9-7cf6-4988-bdfc-020548e04327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:21.753427 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ac4b9-7cf6-4988-bdfc-020548e04327 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:21.753712 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ac4b9-7cf6-4988-bdfc-020548e04327 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:21.754047 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ac4b9-7cf6-4988-bdfc-020548e04327 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:21.754837 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ac4b9-7cf6-4988-bdfc-020548e04327 None admin] RBAC: Adding query filter params (role.id=0cdb52ffa00449399b27dbfcd383f488, scope.project.id=4c7185cb4791495281d3a16293793beb, user.id=ab58de7f803c4cd29f61931966b168d5) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:21.755153 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ac4b9-7cf6-4988-bdfc-020548e04327 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:21.756726 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ac4b9-7cf6-4988-bdfc-020548e04327 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:21.761654 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:21.767725 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 45/94] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:23:21 2022] GET /identity/v3/role_assignments?user.id=ab58de7f803c4cd29f61931966b168d5&scope.project.id=4c7185cb4791495281d3a16293793beb&role.id=0cdb52ffa00449399b27dbfcd383f488 => generated 259 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:21.910210 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666fb2d8-6c4a-4abc-9a42-bcf1971a89e2 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:21.962249 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f471124-8ab1-4c6e-ada8-3af363a0376b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:21.962788 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f471124-8ab1-4c6e-ada8-3af363a0376b None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:21.963079 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f471124-8ab1-4c6e-ada8-3af363a0376b None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:21.963358 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f471124-8ab1-4c6e-ada8-3af363a0376b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:22.066336 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f471124-8ab1-4c6e-ada8-3af363a0376b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:22.067852 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f471124-8ab1-4c6e-ada8-3af363a0376b None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:22.084227 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:22.278605 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 50/95] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:23:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:23:22.278867 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ac4b9-7cf6-4988-bdfc-020548e04327 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:22.418678 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ac4b9-7cf6-4988-bdfc-020548e04327 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:22.460066 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3bb1e9-1f25-43b2-bd0a-383170379c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:22.460746 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3bb1e9-1f25-43b2-bd0a-383170379c1e None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:22.461036 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3bb1e9-1f25-43b2-bd0a-383170379c1e None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:22.461316 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3bb1e9-1f25-43b2-bd0a-383170379c1e None admin] PATH_INFO: `/v3/projects/f2dcd698555f4de580cb7ea6b098dbd7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:22.462473 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3bb1e9-1f25-43b2-bd0a-383170379c1e None admin] RBAC: Authorizing `identity:get_project(project_id=f2dcd698555f4de580cb7ea6b098dbd7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:22.464112 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3bb1e9-1f25-43b2-bd0a-383170379c1e None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:22.466179 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 46/96] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:23:22 2022] GET /identity/v3/projects/f2dcd698555f4de580cb7ea6b098dbd7 => generated 312 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:22.472344 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f471124-8ab1-4c6e-ada8-3af363a0376b None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:22.614679 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f471124-8ab1-4c6e-ada8-3af363a0376b None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:22.625185 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:22.652390 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c080c1ec-cf6e-42ce-86f6-7961811242ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:22.653081 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080c1ec-cf6e-42ce-86f6-7961811242ae None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:22.653369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080c1ec-cf6e-42ce-86f6-7961811242ae None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:22.653652 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080c1ec-cf6e-42ce-86f6-7961811242ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:22.654395 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c080c1ec-cf6e-42ce-86f6-7961811242ae None admin] RBAC: Adding query filter params (role.id=87862cd6eaeb4d5fae089a801aac4f87, scope.project.id=f2dcd698555f4de580cb7ea6b098dbd7, user.id=ab58de7f803c4cd29f61931966b168d5) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:22.654714 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c080c1ec-cf6e-42ce-86f6-7961811242ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:22.656519 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c080c1ec-cf6e-42ce-86f6-7961811242ae None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:22.669834 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 51/97] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:23:22 2022] GET /identity/v3/role_assignments?user.id=ab58de7f803c4cd29f61931966b168d5&scope.project.id=f2dcd698555f4de580cb7ea6b098dbd7&role.id=87862cd6eaeb4d5fae089a801aac4f87 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:24.428874 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731cca2d-6668-47a3-aabc-8507b20691b7 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:24.429397 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731cca2d-6668-47a3-aabc-8507b20691b7 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:24.429397 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731cca2d-6668-47a3-aabc-8507b20691b7 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:24.430832 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 47/98] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:24.436369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3f256-d169-42d6-8c2e-a7a7645f2791 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:24.437241 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3f256-d169-42d6-8c2e-a7a7645f2791 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:24.437724 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3f256-d169-42d6-8c2e-a7a7645f2791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:24.458043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-1af3f256-d169-42d6-8c2e-a7a7645f2791 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:24.459588 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-1af3f256-d169-42d6-8c2e-a7a7645f2791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:24.545623 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 52/99] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:24.565921 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923890a-b2b5-4fc9-a98d-b79680a24445 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:24.566551 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923890a-b2b5-4fc9-a98d-b79680a24445 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:24.567060 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923890a-b2b5-4fc9-a98d-b79680a24445 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:24.568626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 48/100] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:23:24.574278 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0477182-221b-49e2-b881-01bc21ff45e0 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:24.574805 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0477182-221b-49e2-b881-01bc21ff45e0 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:24.575285 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0477182-221b-49e2-b881-01bc21ff45e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:24.601652 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-d0477182-221b-49e2-b881-01bc21ff45e0 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:24.603175 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-d0477182-221b-49e2-b881-01bc21ff45e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:24.621130 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc02a091-9820-419d-b241-d76703e938f5 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:24.621732 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc02a091-9820-419d-b241-d76703e938f5 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:24.622237 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc02a091-9820-419d-b241-d76703e938f5 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:24.623762 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 49/101] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:23:24.629724 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb2690-0fc4-4ebc-94e1-64d8b787b364 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:24.630381 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb2690-0fc4-4ebc-94e1-64d8b787b364 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:24.630886 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb2690-0fc4-4ebc-94e1-64d8b787b364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:24.655939 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-f4cb2690-0fc4-4ebc-94e1-64d8b787b364 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:24.657691 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-f4cb2690-0fc4-4ebc-94e1-64d8b787b364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:24.750831 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 53/102] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:24.784825 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:24.796391 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 50/103] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:23:24.870809 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25b92f-8621-457f-a146-72fac1c9f10b None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:24.871159 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25b92f-8621-457f-a146-72fac1c9f10b None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:24.871378 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25b92f-8621-457f-a146-72fac1c9f10b None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:24.872773 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 54/104] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:24.878861 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c129b-5265-4b98-ba6a-00a295ef5f78 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:24.879160 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c129b-5265-4b98-ba6a-00a295ef5f78 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:24.879449 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c129b-5265-4b98-ba6a-00a295ef5f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:24.916924 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-b09c129b-5265-4b98-ba6a-00a295ef5f78 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:24.918143 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-b09c129b-5265-4b98-ba6a-00a295ef5f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:25.006216 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 51/105] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:25.022070 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea1b6e-1e2c-4033-9f21-71e9e4e1f86e None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:25.022374 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea1b6e-1e2c-4033-9f21-71e9e4e1f86e None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:25.022636 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea1b6e-1e2c-4033-9f21-71e9e4e1f86e None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:25.024309 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 55/106] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:25.026396 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c129b-5265-4b98-ba6a-00a295ef5f78 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:25.029991 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c1f81-4a49-4350-bce9-930400aa45e1 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:25.030789 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c1f81-4a49-4350-bce9-930400aa45e1 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:25.031341 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c1f81-4a49-4350-bce9-930400aa45e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:25.051016 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-396c1f81-4a49-4350-bce9-930400aa45e1 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:25.052612 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-396c1f81-4a49-4350-bce9-930400aa45e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:25.116755 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:25.195977 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 56/107] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:23:25.201579 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c1f81-4a49-4350-bce9-930400aa45e1 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:25.229587 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c129b-5265-4b98-ba6a-00a295ef5f78 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:25.251023 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:25.276253 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e4262-c030-4147-b047-8f7cc977c6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:25.276853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e4262-c030-4147-b047-8f7cc977c6e9 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:25.277145 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e4262-c030-4147-b047-8f7cc977c6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:25.277430 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e4262-c030-4147-b047-8f7cc977c6e9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:25.278570 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e4262-c030-4147-b047-8f7cc977c6e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:25.280187 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e4262-c030-4147-b047-8f7cc977c6e9 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:25.282525 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 52/108] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:23:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:25.283480 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e4262-c030-4147-b047-8f7cc977c6e9 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:25.398745 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c1f81-4a49-4350-bce9-930400aa45e1 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:25.415399 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e4262-c030-4147-b047-8f7cc977c6e9 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:25.446450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b17bf-a390-4707-8160-87d3f4900e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:25.447264 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b17bf-a390-4707-8160-87d3f4900e29 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:25.448170 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b17bf-a390-4707-8160-87d3f4900e29 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:25.448959 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b17bf-a390-4707-8160-87d3f4900e29 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:25.453013 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a25e447-35c9-4a25-bb28-e90e28f01405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:25.458878 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a25e447-35c9-4a25-bb28-e90e28f01405 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:25.458878 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a25e447-35c9-4a25-bb28-e90e28f01405 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:25.458878 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a25e447-35c9-4a25-bb28-e90e28f01405 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:25.465688 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245b17bf-a390-4707-8160-87d3f4900e29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:23:25.470613 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-245b17bf-a390-4707-8160-87d3f4900e29 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:23:25.470613 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:23:25.470613 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:23:25.470613 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:23:25.470613 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:23:25.470613 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:23:25.470613 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:23:25.470613 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:23:25.470613 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:23:25.470613 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:23:25.470613 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:23:25.470613 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:23:25.470613 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:23:25.470613 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:23:25.470613 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:23:25.470613 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:23:25.471670 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:23:25.471670 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:23:25.471670 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:23:25.471670 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:23:25.471670 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:23:25.471670 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:23:25.471670 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:23:25.471670 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:23:25.471670 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:23:25.471670 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:23:25.471670 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:23:25.471670 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:23:25.471670 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:23:25.471670 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:23:25.471670 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:23:25.471670 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:23:25.472522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:23:25.472522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:23:25.472522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:23:25.472522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:23:25.472522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:23:25.472522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:23:25.472522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:23:25.472522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:23:25.472522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:23:25.472522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:23:25.472522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:23:25.472522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:23:25.472522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 57/109] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:23:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:23:25.473187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b17bf-a390-4707-8160-87d3f4900e29 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:25.474207 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a25e447-35c9-4a25-bb28-e90e28f01405 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:23:25.481497 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-7a25e447-35c9-4a25-bb28-e90e28f01405 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:23:25.481497 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:23:25.481497 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:23:25.481497 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:23:25.481497 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:23:25.481497 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:23:25.481497 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:23:25.481497 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:23:25.481497 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:23:25.481497 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:23:25.481497 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:23:25.481497 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:23:25.481497 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:23:25.481497 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:23:25.481497 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:23:25.481497 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:23:25.482370 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:23:25.482370 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:23:25.482370 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:23:25.482370 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:23:25.482370 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:23:25.482370 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:23:25.482370 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:23:25.482370 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:23:25.482370 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:23:25.482370 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:23:25.482370 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:23:25.482370 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:23:25.482370 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:23:25.482370 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:23:25.482370 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:23:25.482370 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:23:25.483232 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:23:25.483232 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:23:25.483232 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:23:25.483232 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:23:25.483232 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:23:25.483232 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:23:25.483232 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:23:25.483232 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:23:25.483232 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:23:25.483232 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:23:25.483232 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:23:25.483232 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:23:25.487403 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 53/110] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Aug 24 05:23:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:23:25.493348 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a25e447-35c9-4a25-bb28-e90e28f01405 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:25.504304 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:25.551535 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:25.583960 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b17bf-a390-4707-8160-87d3f4900e29 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:25.606396 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a25e447-35c9-4a25-bb28-e90e28f01405 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:25.619081 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e06291-3dba-483f-b6fe-ae5878868109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:25.619716 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e06291-3dba-483f-b6fe-ae5878868109 None admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:25.620008 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e06291-3dba-483f-b6fe-ae5878868109 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:25.620292 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e06291-3dba-483f-b6fe-ae5878868109 None admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:25.620985 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e06291-3dba-483f-b6fe-ae5878868109 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:25.622504 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e06291-3dba-483f-b6fe-ae5878868109 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:25.640435 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f09a8f9-3793-4186-8add-55311fcf2909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:25.642470 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09a8f9-3793-4186-8add-55311fcf2909 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:25.642470 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09a8f9-3793-4186-8add-55311fcf2909 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:25.642470 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09a8f9-3793-4186-8add-55311fcf2909 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:25.652838 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f09a8f9-3793-4186-8add-55311fcf2909 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:23:25.658483 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-1f09a8f9-3793-4186-8add-55311fcf2909 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:23:25.658483 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:23:25.658483 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:23:25.658483 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:23:25.658483 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:23:25.658483 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:23:25.658483 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:23:25.658483 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:23:25.658483 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:23:25.658483 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:23:25.658483 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:23:25.658483 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:23:25.658483 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:23:25.658483 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:23:25.658483 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:23:25.658483 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:23:25.664739 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:23:25.664739 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:23:25.664739 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:23:25.664739 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:23:25.664739 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:23:25.664739 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:23:25.664739 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:23:25.664739 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:23:25.664739 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:23:25.664739 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:23:25.664739 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:23:25.664739 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:23:25.664739 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:23:25.664739 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:23:25.664739 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:23:25.664739 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:23:25.665539 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:23:25.665539 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:23:25.665539 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:23:25.665539 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:23:25.665539 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:23:25.665539 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:23:25.665539 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:23:25.665539 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:23:25.665539 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:23:25.665539 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:23:25.665539 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:23:25.665539 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:23:25.665539 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 54/111] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:23:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:23:25.665539 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f09a8f9-3793-4186-8add-55311fcf2909 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:25.735252 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 58/112] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Aug 24 05:23:25 2022] POST /identity/v3/projects => generated 302 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:23:25.736896 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e06291-3dba-483f-b6fe-ae5878868109 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:25.794696 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f09a8f9-3793-4186-8add-55311fcf2909 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:25.802702 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:25.837244 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:25.840947 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e12487-4157-495a-b10b-dc03ea566503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:25.842102 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e12487-4157-495a-b10b-dc03ea566503 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:25.842622 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e12487-4157-495a-b10b-dc03ea566503 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:25.843156 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e12487-4157-495a-b10b-dc03ea566503 None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:25.844551 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e12487-4157-495a-b10b-dc03ea566503 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:25.845144 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e12487-4157-495a-b10b-dc03ea566503 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:25.847035 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e12487-4157-495a-b10b-dc03ea566503 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:25.856498 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 55/113] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Aug 24 05:23:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:25.857380 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e12487-4157-495a-b10b-dc03ea566503 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:25.870607 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e06291-3dba-483f-b6fe-ae5878868109 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:25.919071 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e5c9c-21a3-4412-8b78-050e5fdf792f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:25.919702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e5c9c-21a3-4412-8b78-050e5fdf792f None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:25.919989 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e5c9c-21a3-4412-8b78-050e5fdf792f None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:25.920264 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e5c9c-21a3-4412-8b78-050e5fdf792f None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:25.920972 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e5c9c-21a3-4412-8b78-050e5fdf792f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:25.921402 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e5c9c-21a3-4412-8b78-050e5fdf792f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:25.922796 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e5c9c-21a3-4412-8b78-050e5fdf792f None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:25.932171 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 59/114] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:23:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:25.933081 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e5c9c-21a3-4412-8b78-050e5fdf792f None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:25.978873 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e12487-4157-495a-b10b-dc03ea566503 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:26.016251 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f316d5e-79be-4be5-8cc8-da77f66036bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:26.017036 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f316d5e-79be-4be5-8cc8-da77f66036bb None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:26.017455 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f316d5e-79be-4be5-8cc8-da77f66036bb None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:26.017858 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f316d5e-79be-4be5-8cc8-da77f66036bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:26.028845 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f316d5e-79be-4be5-8cc8-da77f66036bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:23:26.035525 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-6f316d5e-79be-4be5-8cc8-da77f66036bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:23:26.035525 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:23:26.035525 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:23:26.035525 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:23:26.035525 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:23:26.035525 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:23:26.035525 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:23:26.035525 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:23:26.035525 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:23:26.035525 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:23:26.035525 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:23:26.035525 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:23:26.035525 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:23:26.035525 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:23:26.035525 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:23:26.035525 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:23:26.036492 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:23:26.036492 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:23:26.036492 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:23:26.036492 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:23:26.036492 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:23:26.036492 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:23:26.036492 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:23:26.036492 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:23:26.036492 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:23:26.036492 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:23:26.036492 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:23:26.036492 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:23:26.036492 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:23:26.036492 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:23:26.036492 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:23:26.036492 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:23:26.037605 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:23:26.037605 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:23:26.037605 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:23:26.037605 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:23:26.037605 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:23:26.037605 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:23:26.037605 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:23:26.037605 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:23:26.037605 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:23:26.037605 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:23:26.037605 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:23:26.037605 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:23:26.037605 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 56/115] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Aug 24 05:23:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:23:26.038342 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a2241-4b2f-413f-8ebb-5d09f3c28fea None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:26.038342 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a2241-4b2f-413f-8ebb-5d09f3c28fea None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:26.039594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a2241-4b2f-413f-8ebb-5d09f3c28fea None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:26.041084 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 57/116] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:26.041724 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a2241-4b2f-413f-8ebb-5d09f3c28fea None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:26.053847 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e5c9c-21a3-4412-8b78-050e5fdf792f None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:26.080308 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:26.101138 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5afe44b-77d1-414a-a4e0-7a5a8929e764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:26.101808 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5afe44b-77d1-414a-a4e0-7a5a8929e764 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:26.102166 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5afe44b-77d1-414a-a4e0-7a5a8929e764 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:26.102518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5afe44b-77d1-414a-a4e0-7a5a8929e764 None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:26.103315 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5afe44b-77d1-414a-a4e0-7a5a8929e764 None admin] RBAC: Adding query filter params (name=member) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:26.103712 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5afe44b-77d1-414a-a4e0-7a5a8929e764 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:26.105673 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5afe44b-77d1-414a-a4e0-7a5a8929e764 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:26.108714 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:26.116250 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 60/117] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Aug 24 05:23:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:26.117270 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5afe44b-77d1-414a-a4e0-7a5a8929e764 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:26.174729 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a2241-4b2f-413f-8ebb-5d09f3c28fea None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:26.215800 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c6e5c-d1ea-49fa-9c6c-d24f4a51bacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:26.216463 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c6e5c-d1ea-49fa-9c6c-d24f4a51bacf None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:26.216746 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c6e5c-d1ea-49fa-9c6c-d24f4a51bacf None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:26.217021 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c6e5c-d1ea-49fa-9c6c-d24f4a51bacf None admin] PATH_INFO: `/v3/users/2c997665052f41d1b074897ce9efcc3d` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:26.228959 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5afe44b-77d1-414a-a4e0-7a5a8929e764 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:26.231180 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c6e5c-d1ea-49fa-9c6c-d24f4a51bacf None admin] RBAC: Authorizing `identity:get_user(user_id=2c997665052f41d1b074897ce9efcc3d)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:26.232683 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c6e5c-d1ea-49fa-9c6c-d24f4a51bacf None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:26.234550 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 58/118] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:23:26 2022] GET /identity/v3/users/2c997665052f41d1b074897ce9efcc3d => generated 289 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:26.235233 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c6e5c-d1ea-49fa-9c6c-d24f4a51bacf None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:26.273495 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f7842b-8315-441b-abd8-c8ef43bd5e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:26.274506 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f7842b-8315-441b-abd8-c8ef43bd5e2f None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:26.274995 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f7842b-8315-441b-abd8-c8ef43bd5e2f None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:26.275507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f7842b-8315-441b-abd8-c8ef43bd5e2f None admin] PATH_INFO: `/v3/users/ab58de7f803c4cd29f61931966b168d5` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:26.276910 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f7842b-8315-441b-abd8-c8ef43bd5e2f None admin] RBAC: Authorizing `identity:get_user(user_id=ab58de7f803c4cd29f61931966b168d5)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:26.278468 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f7842b-8315-441b-abd8-c8ef43bd5e2f None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:26.280244 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 61/119] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:23:26 2022] GET /identity/v3/users/ab58de7f803c4cd29f61931966b168d5 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:26.281535 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c1e4a-4372-4b4d-acb1-171d608d4956 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:26.281818 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c1e4a-4372-4b4d-acb1-171d608d4956 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:26.282103 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c1e4a-4372-4b4d-acb1-171d608d4956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:26.313692 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-f11c1e4a-4372-4b4d-acb1-171d608d4956 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:26.315396 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-f11c1e4a-4372-4b4d-acb1-171d608d4956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:26.325984 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:26.341529 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c6e5c-d1ea-49fa-9c6c-d24f4a51bacf None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:26.382109 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e36079b-e795-4090-9132-05e2c5622416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:26.382544 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:26.383698 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e36079b-e795-4090-9132-05e2c5622416 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:26.384141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e36079b-e795-4090-9132-05e2c5622416 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:26.384439 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e36079b-e795-4090-9132-05e2c5622416 None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:26.385126 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e36079b-e795-4090-9132-05e2c5622416 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:26.385435 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e36079b-e795-4090-9132-05e2c5622416 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:26.386842 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e36079b-e795-4090-9132-05e2c5622416 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:26.390368 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 62/120] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:26.399911 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 59/121] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:23:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:26.401343 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e36079b-e795-4090-9132-05e2c5622416 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:26.401449 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c1e4a-4372-4b4d-acb1-171d608d4956 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:26.515117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e36079b-e795-4090-9132-05e2c5622416 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:26.524383 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c1e4a-4372-4b4d-acb1-171d608d4956 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:26.552239 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ec1b4-0d56-40b0-8d44-cdfa3c078755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:26.553253 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ec1b4-0d56-40b0-8d44-cdfa3c078755 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:26.553847 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ec1b4-0d56-40b0-8d44-cdfa3c078755 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:26.554316 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ec1b4-0d56-40b0-8d44-cdfa3c078755 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:26.569137 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83446690-9900-46cb-b182-a10bc8a65bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:26.569917 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83446690-9900-46cb-b182-a10bc8a65bfb None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:26.570212 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83446690-9900-46cb-b182-a10bc8a65bfb None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:26.570601 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83446690-9900-46cb-b182-a10bc8a65bfb None admin] PATH_INFO: `/v3/users/ab58de7f803c4cd29f61931966b168d5` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:26.572529 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83446690-9900-46cb-b182-a10bc8a65bfb None admin] RBAC: Authorizing `identity:get_user(user_id=ab58de7f803c4cd29f61931966b168d5)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:26.575423 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83446690-9900-46cb-b182-a10bc8a65bfb None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:26.577045 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:26.584426 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 63/122] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:23:26 2022] GET /identity/v3/users/ab58de7f803c4cd29f61931966b168d5 => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:26.584426 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83446690-9900-46cb-b182-a10bc8a65bfb None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:26.637424 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88ec1b4-0d56-40b0-8d44-cdfa3c078755 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:26.639345 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88ec1b4-0d56-40b0-8d44-cdfa3c078755 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:26.699401 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:26.747790 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83446690-9900-46cb-b182-a10bc8a65bfb None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:26.764955 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 60/123] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:23:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:23:26.765814 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ec1b4-0d56-40b0-8d44-cdfa3c078755 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:26.793103 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e71caf-9a2d-4a1a-a831-97df5830da68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:26.793758 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e71caf-9a2d-4a1a-a831-97df5830da68 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:26.794045 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e71caf-9a2d-4a1a-a831-97df5830da68 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:26.794326 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e71caf-9a2d-4a1a-a831-97df5830da68 None admin] PATH_INFO: `/v3/projects/4c7185cb4791495281d3a16293793beb` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:26.795481 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e71caf-9a2d-4a1a-a831-97df5830da68 None admin] RBAC: Authorizing `identity:get_project(project_id=4c7185cb4791495281d3a16293793beb)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:26.796991 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e71caf-9a2d-4a1a-a831-97df5830da68 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:26.800066 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 64/124] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:23:26 2022] GET /identity/v3/projects/4c7185cb4791495281d3a16293793beb => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:26.801175 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e71caf-9a2d-4a1a-a831-97df5830da68 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:26.824176 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:26.882288 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ec1b4-0d56-40b0-8d44-cdfa3c078755 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:26.921917 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fa94cd-c3a9-4f78-a544-5b3deba943ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:26.922714 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fa94cd-c3a9-4f78-a544-5b3deba943ac None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:26.923554 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fa94cd-c3a9-4f78-a544-5b3deba943ac None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:26.924043 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fa94cd-c3a9-4f78-a544-5b3deba943ac None admin] PATH_INFO: `/v3/users/359799b7d001453b90d984a05215cce6` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:26.925690 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fa94cd-c3a9-4f78-a544-5b3deba943ac None admin] RBAC: Authorizing `identity:get_user(user_id=359799b7d001453b90d984a05215cce6)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:26.927341 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fa94cd-c3a9-4f78-a544-5b3deba943ac None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:26.928770 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e71caf-9a2d-4a1a-a831-97df5830da68 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:26.928975 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 61/125] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:23:26 2022] GET /identity/v3/users/359799b7d001453b90d984a05215cce6 => generated 247 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:26.930142 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fa94cd-c3a9-4f78-a544-5b3deba943ac None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:26.973945 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:26.978711 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77fe700-cef8-4e7a-b6a0-5dfb203cf816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:26.979369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77fe700-cef8-4e7a-b6a0-5dfb203cf816 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:26.979699 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77fe700-cef8-4e7a-b6a0-5dfb203cf816 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:26.980000 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77fe700-cef8-4e7a-b6a0-5dfb203cf816 None admin] PATH_INFO: `/v3/projects/4c7185cb4791495281d3a16293793beb` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:26.981179 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77fe700-cef8-4e7a-b6a0-5dfb203cf816 None admin] RBAC: Authorizing `identity:get_project(project_id=4c7185cb4791495281d3a16293793beb)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:26.982709 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77fe700-cef8-4e7a-b6a0-5dfb203cf816 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:26.989347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 65/126] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:23:26 2022] GET /identity/v3/projects/4c7185cb4791495281d3a16293793beb => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:26.989347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77fe700-cef8-4e7a-b6a0-5dfb203cf816 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:27.052657 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fa94cd-c3a9-4f78-a544-5b3deba943ac None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:27.071106 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:27.090866 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9047f889-fb64-4db7-b769-e6f244ff0abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:27.091423 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9047f889-fb64-4db7-b769-e6f244ff0abc None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:27.091768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9047f889-fb64-4db7-b769-e6f244ff0abc None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:27.092053 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9047f889-fb64-4db7-b769-e6f244ff0abc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:27.100150 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77fe700-cef8-4e7a-b6a0-5dfb203cf816 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:27.105779 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9047f889-fb64-4db7-b769-e6f244ff0abc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:23:27.111326 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-9047f889-fb64-4db7-b769-e6f244ff0abc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:23:27.111326 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:23:27.111326 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:23:27.111326 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:23:27.111326 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:23:27.111326 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:23:27.111326 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:23:27.111326 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:23:27.111326 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:23:27.111326 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:23:27.111326 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:23:27.111326 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:23:27.111326 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:23:27.111326 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:23:27.111326 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:23:27.111326 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:23:27.112271 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:23:27.112271 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:23:27.112271 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:23:27.112271 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:23:27.112271 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:23:27.112271 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:23:27.112271 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:23:27.112271 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:23:27.112271 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:23:27.112271 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:23:27.112271 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:23:27.112271 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:23:27.112271 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:23:27.112271 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:23:27.112271 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:23:27.112271 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:23:27.113074 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:23:27.113074 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:23:27.113074 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:23:27.113074 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:23:27.113074 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:23:27.113074 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:23:27.113074 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:23:27.113074 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:23:27.113074 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:23:27.113074 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:23:27.113074 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:23:27.113074 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:23:27.113074 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 62/127] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:23:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:23:27.113733 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9047f889-fb64-4db7-b769-e6f244ff0abc None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:27.143364 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cb6de4-4945-4c0e-9387-3d2e1505c616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:27.144505 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb6de4-4945-4c0e-9387-3d2e1505c616 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:27.145010 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb6de4-4945-4c0e-9387-3d2e1505c616 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:27.145529 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb6de4-4945-4c0e-9387-3d2e1505c616 None admin] PATH_INFO: `/v3/projects/4c7185cb4791495281d3a16293793beb` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:27.146994 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cb6de4-4945-4c0e-9387-3d2e1505c616 None admin] RBAC: Authorizing `identity:get_project(project_id=4c7185cb4791495281d3a16293793beb)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:27.148938 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cb6de4-4945-4c0e-9387-3d2e1505c616 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:27.150968 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 66/128] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:23:26 2022] GET /identity/v3/projects/4c7185cb4791495281d3a16293793beb => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:27.151937 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cb6de4-4945-4c0e-9387-3d2e1505c616 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:27.230207 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:27.232253 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9047f889-fb64-4db7-b769-e6f244ff0abc None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:27.268265 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4601da56-a2ab-4162-809d-a0cd556789bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:27.268958 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4601da56-a2ab-4162-809d-a0cd556789bd None admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:27.269240 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4601da56-a2ab-4162-809d-a0cd556789bd None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:27.269523 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4601da56-a2ab-4162-809d-a0cd556789bd None admin] PATH_INFO: `/v3/projects/4c7185cb4791495281d3a16293793beb/users/ab58de7f803c4cd29f61931966b168d5/roles/0cdb52ffa00449399b27dbfcd383f488` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:27.271809 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4601da56-a2ab-4162-809d-a0cd556789bd None admin] RBAC: Authorizing `identity:create_grant(project_id=4c7185cb4791495281d3a16293793beb, user_id=ab58de7f803c4cd29f61931966b168d5, role_id=0cdb52ffa00449399b27dbfcd383f488)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:27.272903 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cb6de4-4945-4c0e-9387-3d2e1505c616 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:27.273298 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4601da56-a2ab-4162-809d-a0cd556789bd None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:27.309573 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d041302d-e9f0-43a9-b2d7-362978fc147c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:27.310214 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041302d-e9f0-43a9-b2d7-362978fc147c None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:27.310497 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041302d-e9f0-43a9-b2d7-362978fc147c None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:27.310774 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041302d-e9f0-43a9-b2d7-362978fc147c None admin] PATH_INFO: `/v3/projects/4c7185cb4791495281d3a16293793beb` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:27.311897 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d041302d-e9f0-43a9-b2d7-362978fc147c None admin] RBAC: Authorizing `identity:get_project(project_id=4c7185cb4791495281d3a16293793beb)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:27.313396 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d041302d-e9f0-43a9-b2d7-362978fc147c None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:27.315711 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 67/129] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:23:27 2022] GET /identity/v3/projects/4c7185cb4791495281d3a16293793beb => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:27.317475 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d041302d-e9f0-43a9-b2d7-362978fc147c None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:27.408206 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 63/130] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Aug 24 05:23:27 2022] PUT /identity/v3/projects/4c7185cb4791495281d3a16293793beb/users/ab58de7f803c4cd29f61931966b168d5/roles/0cdb52ffa00449399b27dbfcd383f488 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:23:27.421388 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4601da56-a2ab-4162-809d-a0cd556789bd None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:27.424115 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d041302d-e9f0-43a9-b2d7-362978fc147c None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:27.474688 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a534aabb-c817-48b9-92ea-6e0e0db59e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:27.475388 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534aabb-c817-48b9-92ea-6e0e0db59e75 None admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:27.475813 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534aabb-c817-48b9-92ea-6e0e0db59e75 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:27.476102 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534aabb-c817-48b9-92ea-6e0e0db59e75 None admin] PATH_INFO: `/v3/projects/4c7185cb4791495281d3a16293793beb/users/ab58de7f803c4cd29f61931966b168d5/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:27.478202 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:27.479703 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a534aabb-c817-48b9-92ea-6e0e0db59e75 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c7185cb4791495281d3a16293793beb, user_id=ab58de7f803c4cd29f61931966b168d5, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:27.481417 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a534aabb-c817-48b9-92ea-6e0e0db59e75 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:27.487998 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:27.557465 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4601da56-a2ab-4162-809d-a0cd556789bd None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:27.600928 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770f8eb-03aa-44d3-94aa-3aa104e9433b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:27.601637 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770f8eb-03aa-44d3-94aa-3aa104e9433b None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:27.603608 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770f8eb-03aa-44d3-94aa-3aa104e9433b None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:27.603608 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770f8eb-03aa-44d3-94aa-3aa104e9433b None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:27.603608 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f770f8eb-03aa-44d3-94aa-3aa104e9433b None admin] RBAC: Adding query filter params (name=member) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:27.603608 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f770f8eb-03aa-44d3-94aa-3aa104e9433b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:27.604559 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f770f8eb-03aa-44d3-94aa-3aa104e9433b None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:27.613458 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 64/131] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Aug 24 05:23:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:27.617788 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770f8eb-03aa-44d3-94aa-3aa104e9433b None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:27.634544 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 68/132] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Aug 24 05:23:27 2022] PUT /identity/v3/projects/4c7185cb4791495281d3a16293793beb/users/ab58de7f803c4cd29f61931966b168d5/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:23:27.641733 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a534aabb-c817-48b9-92ea-6e0e0db59e75 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:27.781195 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770f8eb-03aa-44d3-94aa-3aa104e9433b None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:27.784377 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:27.834030 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d6700c-f58f-42b0-a34e-3405c622be67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:27.835017 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d6700c-f58f-42b0-a34e-3405c622be67 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:27.835501 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d6700c-f58f-42b0-a34e-3405c622be67 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:27.835924 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d6700c-f58f-42b0-a34e-3405c622be67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:27.836809 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d6700c-f58f-42b0-a34e-3405c622be67 None admin] RBAC: Adding query filter params (role.id=49b52d60e4434f05870a37c49b2e6e6b, scope.project.id=4c7185cb4791495281d3a16293793beb, user.id=2c997665052f41d1b074897ce9efcc3d) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:27.837196 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d6700c-f58f-42b0-a34e-3405c622be67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:27.839451 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d6700c-f58f-42b0-a34e-3405c622be67 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:27.845934 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 65/133] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:23:27 2022] GET /identity/v3/role_assignments?user.id=2c997665052f41d1b074897ce9efcc3d&scope.project.id=4c7185cb4791495281d3a16293793beb&role.id=49b52d60e4434f05870a37c49b2e6e6b => generated 259 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:27.847034 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d6700c-f58f-42b0-a34e-3405c622be67 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:27.863148 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a534aabb-c817-48b9-92ea-6e0e0db59e75 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:27.906941 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cdc5da-05f2-4395-819c-8c69bae2facd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:27.908540 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdc5da-05f2-4395-819c-8c69bae2facd None admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:27.909016 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdc5da-05f2-4395-819c-8c69bae2facd None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:27.909310 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdc5da-05f2-4395-819c-8c69bae2facd None admin] PATH_INFO: `/v3/projects/4c7185cb4791495281d3a16293793beb/users/359799b7d001453b90d984a05215cce6/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:27.911566 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cdc5da-05f2-4395-819c-8c69bae2facd None admin] RBAC: Authorizing `identity:create_grant(project_id=4c7185cb4791495281d3a16293793beb, user_id=359799b7d001453b90d984a05215cce6, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:27.913194 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cdc5da-05f2-4395-819c-8c69bae2facd None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:27.982921 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d6700c-f58f-42b0-a34e-3405c622be67 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:27.992521 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 69/134] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Aug 24 05:23:27 2022] PUT /identity/v3/projects/4c7185cb4791495281d3a16293793beb/users/359799b7d001453b90d984a05215cce6/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:23:27.994506 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502dc4c8-939b-45b5-9bef-dc46ddc00566 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:27.994799 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502dc4c8-939b-45b5-9bef-dc46ddc00566 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:27.995133 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502dc4c8-939b-45b5-9bef-dc46ddc00566 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:27.997902 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 70/135] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:28.004833 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab38c1-7b9c-4770-94c3-d7f817f0deb6 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:28.005153 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab38c1-7b9c-4770-94c3-d7f817f0deb6 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:28.005431 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab38c1-7b9c-4770-94c3-d7f817f0deb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:28.039651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-42ab38c1-7b9c-4770-94c3-d7f817f0deb6 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:28.041539 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-42ab38c1-7b9c-4770-94c3-d7f817f0deb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:28.067927 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efa5725-e9e1-4270-832b-182490f37073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:28.069150 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa5725-e9e1-4270-832b-182490f37073 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:28.069150 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa5725-e9e1-4270-832b-182490f37073 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:28.069150 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa5725-e9e1-4270-832b-182490f37073 None admin] PATH_INFO: `/v3/users/ab58de7f803c4cd29f61931966b168d5` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:28.070489 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efa5725-e9e1-4270-832b-182490f37073 None admin] RBAC: Authorizing `identity:get_user(user_id=ab58de7f803c4cd29f61931966b168d5)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:28.072288 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efa5725-e9e1-4270-832b-182490f37073 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:28.074203 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 66/136] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:23:27 2022] GET /identity/v3/users/ab58de7f803c4cd29f61931966b168d5 => generated 275 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:28.081628 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efa5725-e9e1-4270-832b-182490f37073 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:28.132901 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:28.144997 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:28.214726 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 71/137] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:23:28.279229 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efa5725-e9e1-4270-832b-182490f37073 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:28.323475 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b23f7-bef8-47cc-86f9-8d6ba429c737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:28.324046 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b23f7-bef8-47cc-86f9-8d6ba429c737 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:28.324332 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b23f7-bef8-47cc-86f9-8d6ba429c737 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:28.324640 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b23f7-bef8-47cc-86f9-8d6ba429c737 None admin] PATH_INFO: `/v3/projects/f2dcd698555f4de580cb7ea6b098dbd7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:28.329234 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b23f7-bef8-47cc-86f9-8d6ba429c737 None admin] RBAC: Authorizing `identity:get_project(project_id=f2dcd698555f4de580cb7ea6b098dbd7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:28.330790 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b23f7-bef8-47cc-86f9-8d6ba429c737 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:28.332516 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 67/138] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:23:28 2022] GET /identity/v3/projects/f2dcd698555f4de580cb7ea6b098dbd7 => generated 312 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:28.338601 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab38c1-7b9c-4770-94c3-d7f817f0deb6 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:28.479352 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab38c1-7b9c-4770-94c3-d7f817f0deb6 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:28.531818 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d2038-0e36-4626-8236-306d4070b148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:28.533556 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d2038-0e36-4626-8236-306d4070b148 None admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:28.534036 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d2038-0e36-4626-8236-306d4070b148 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:28.534506 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d2038-0e36-4626-8236-306d4070b148 None admin] PATH_INFO: `/v3/projects/f2dcd698555f4de580cb7ea6b098dbd7/users/ab58de7f803c4cd29f61931966b168d5/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:28.536878 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72d2038-0e36-4626-8236-306d4070b148 None admin] RBAC: Authorizing `identity:create_grant(project_id=f2dcd698555f4de580cb7ea6b098dbd7, user_id=ab58de7f803c4cd29f61931966b168d5, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:28.538639 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72d2038-0e36-4626-8236-306d4070b148 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:28.632537 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 72/139] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Aug 24 05:23:28 2022] PUT /identity/v3/projects/f2dcd698555f4de580cb7ea6b098dbd7/users/ab58de7f803c4cd29f61931966b168d5/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:23:28.680249 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b23f7-bef8-47cc-86f9-8d6ba429c737 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:28.825578 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:28.830351 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b23f7-bef8-47cc-86f9-8d6ba429c737 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:28.875905 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff1a9d7-b6c0-49bc-8351-22e087235bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:28.877362 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff1a9d7-b6c0-49bc-8351-22e087235bc9 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:28.877808 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff1a9d7-b6c0-49bc-8351-22e087235bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:28.878427 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff1a9d7-b6c0-49bc-8351-22e087235bc9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:28.879650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff1a9d7-b6c0-49bc-8351-22e087235bc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:28.881162 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff1a9d7-b6c0-49bc-8351-22e087235bc9 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:28.882841 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 68/140] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:23:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:28.888725 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d2038-0e36-4626-8236-306d4070b148 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:28.961812 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:29.021372 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d2038-0e36-4626-8236-306d4070b148 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:29.066637 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d6c92e-fb76-40d0-903c-01aa8c77d4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:29.067260 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d6c92e-fb76-40d0-903c-01aa8c77d4af None admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:29.067548 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d6c92e-fb76-40d0-903c-01aa8c77d4af None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:29.067884 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d6c92e-fb76-40d0-903c-01aa8c77d4af None admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:29.068578 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d6c92e-fb76-40d0-903c-01aa8c77d4af None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:29.070019 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d6c92e-fb76-40d0-903c-01aa8c77d4af None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:29.197665 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 73/141] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Aug 24 05:23:28 2022] POST /identity/v3/users => generated 283 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:23:30.038111 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78b91d-4b6e-40e5-92c5-9abd81391ec4 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:30.038610 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78b91d-4b6e-40e5-92c5-9abd81391ec4 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:30.038681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78b91d-4b6e-40e5-92c5-9abd81391ec4 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:30.040138 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 69/142] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:30.045692 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d443f-8ed9-4bc3-b4e1-6e490de5eb0c None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:30.046514 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d443f-8ed9-4bc3-b4e1-6e490de5eb0c None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:30.047100 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d443f-8ed9-4bc3-b4e1-6e490de5eb0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:30.067724 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-fe9d443f-8ed9-4bc3-b4e1-6e490de5eb0c None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:30.069330 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-fe9d443f-8ed9-4bc3-b4e1-6e490de5eb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:30.152975 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 74/143] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:30.324790 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7142f63-1c72-412f-b6a0-21faaf2cc7bf None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:30.325384 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7142f63-1c72-412f-b6a0-21faaf2cc7bf None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:30.325866 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7142f63-1c72-412f-b6a0-21faaf2cc7bf None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:30.327419 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 70/144] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:30.333322 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa61f9-c87a-448d-9d4a-524d055d8fa9 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:30.333894 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa61f9-c87a-448d-9d4a-524d055d8fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:30.334363 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa61f9-c87a-448d-9d4a-524d055d8fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:30.355606 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-41fa61f9-c87a-448d-9d4a-524d055d8fa9 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:30.357153 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-41fa61f9-c87a-448d-9d4a-524d055d8fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:30.388426 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:30.433781 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 75/145] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:30.461064 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31a1e5-9bf4-40c0-89da-db2cc694fa55 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:30.461383 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31a1e5-9bf4-40c0-89da-db2cc694fa55 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:30.461632 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31a1e5-9bf4-40c0-89da-db2cc694fa55 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:30.463184 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 71/146] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:30.472978 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb30ad-3794-4770-ad2c-be0354c7e0cd None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:30.473269 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb30ad-3794-4770-ad2c-be0354c7e0cd None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:30.473696 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb30ad-3794-4770-ad2c-be0354c7e0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:30.493145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-8ccb30ad-3794-4770-ad2c-be0354c7e0cd None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:30.494384 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-8ccb30ad-3794-4770-ad2c-be0354c7e0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:30.572471 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 76/147] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:30.622174 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c31a1e5-9bf4-40c0-89da-db2cc694fa55 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:30.749378 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c31a1e5-9bf4-40c0-89da-db2cc694fa55 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:30.792879 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a2843-253c-43a1-a8cb-5068734d3b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:30.793518 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a2843-253c-43a1-a8cb-5068734d3b99 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:30.793802 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a2843-253c-43a1-a8cb-5068734d3b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:30.794080 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a2843-253c-43a1-a8cb-5068734d3b99 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:30.806871 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794a2843-253c-43a1-a8cb-5068734d3b99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:23:30.811724 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-794a2843-253c-43a1-a8cb-5068734d3b99 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:23:30.811724 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:23:30.811724 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:23:30.811724 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:23:30.811724 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:23:30.811724 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:23:30.811724 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:23:30.811724 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:23:30.811724 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:23:30.811724 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:23:30.811724 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:23:30.811724 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:23:30.811724 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:23:30.811724 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:23:30.811724 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:23:30.811724 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:23:30.812598 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:23:30.812598 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:23:30.812598 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:23:30.812598 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:23:30.812598 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:23:30.812598 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:23:30.812598 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:23:30.812598 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:23:30.812598 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:23:30.812598 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:23:30.812598 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:23:30.812598 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:23:30.812598 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:23:30.812598 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:23:30.812598 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:23:30.812598 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:23:30.813347 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:23:30.813347 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:23:30.813347 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:23:30.813347 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:23:30.813347 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:23:30.813347 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:23:30.813347 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:23:30.813347 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:23:30.813347 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:23:30.813347 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:23:30.813347 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:23:30.813347 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:23:30.813347 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 72/148] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Aug 24 05:23:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:23:30.819077 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccb30ad-3794-4770-ad2c-be0354c7e0cd None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:30.839983 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a2843-253c-43a1-a8cb-5068734d3b99 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:30.879150 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:30.968248 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a2843-253c-43a1-a8cb-5068734d3b99 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:31.014761 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac9cdc6-ab0f-489a-9b3d-69744f3fb504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:31.016189 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac9cdc6-ab0f-489a-9b3d-69744f3fb504 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:31.016733 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac9cdc6-ab0f-489a-9b3d-69744f3fb504 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:31.017483 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac9cdc6-ab0f-489a-9b3d-69744f3fb504 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:31.026708 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:31.032088 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac9cdc6-ab0f-489a-9b3d-69744f3fb504 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:23:31.042267 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-bac9cdc6-ab0f-489a-9b3d-69744f3fb504 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:23:31.042267 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:23:31.042267 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:23:31.042267 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:23:31.042267 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:23:31.042267 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:23:31.042267 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:23:31.042267 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:23:31.042267 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:23:31.042267 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:23:31.042267 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:23:31.042267 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:23:31.042267 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:23:31.042267 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:23:31.042267 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:23:31.042267 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:23:31.043758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:23:31.043758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:23:31.043758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:23:31.043758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:23:31.043758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:23:31.043758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:23:31.043758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:23:31.043758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:23:31.043758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:23:31.043758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:23:31.043758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:23:31.043758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:23:31.043758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:23:31.043758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:23:31.043758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:23:31.043758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:23:31.044782 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:23:31.044782 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:23:31.044782 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:23:31.044782 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:23:31.044782 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:23:31.044782 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:23:31.044782 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:23:31.044782 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:23:31.044782 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:23:31.044782 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:23:31.044782 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:23:31.044782 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:23:31.049248 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccb30ad-3794-4770-ad2c-be0354c7e0cd None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:31.053115 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 73/149] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:23:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:23:31.060110 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac9cdc6-ab0f-489a-9b3d-69744f3fb504 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:31.108175 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ccd641-4eaa-44eb-bd77-21740652e91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:31.109330 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ccd641-4eaa-44eb-bd77-21740652e91c None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:31.109778 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ccd641-4eaa-44eb-bd77-21740652e91c None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:31.110199 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ccd641-4eaa-44eb-bd77-21740652e91c None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:31.111920 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ccd641-4eaa-44eb-bd77-21740652e91c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:31.112437 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ccd641-4eaa-44eb-bd77-21740652e91c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:31.114217 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ccd641-4eaa-44eb-bd77-21740652e91c None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:31.126376 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 77/150] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:23:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:31.135611 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ccd641-4eaa-44eb-bd77-21740652e91c None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:31.238481 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:31.255025 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac9cdc6-ab0f-489a-9b3d-69744f3fb504 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:31.300707 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39dcdc-f9e4-4fbd-815a-121bf261f924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:31.301553 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39dcdc-f9e4-4fbd-815a-121bf261f924 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:31.301975 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39dcdc-f9e4-4fbd-815a-121bf261f924 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:31.302412 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39dcdc-f9e4-4fbd-815a-121bf261f924 None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:31.303274 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a39dcdc-f9e4-4fbd-815a-121bf261f924 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:31.303793 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a39dcdc-f9e4-4fbd-815a-121bf261f924 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:31.305379 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a39dcdc-f9e4-4fbd-815a-121bf261f924 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:31.309399 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ccd641-4eaa-44eb-bd77-21740652e91c None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:31.315418 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 74/151] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Aug 24 05:23:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:23:31.317270 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9532746-521e-4ea4-adf7-5c771d3d8ff9 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:31.317738 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9532746-521e-4ea4-adf7-5c771d3d8ff9 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:31.318243 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9532746-521e-4ea4-adf7-5c771d3d8ff9 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:31.320019 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 75/152] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:31.321062 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9532746-521e-4ea4-adf7-5c771d3d8ff9 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:31.367441 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87efca8-73d0-4b26-84a0-b4909e60047a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:31.369066 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87efca8-73d0-4b26-84a0-b4909e60047a None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:31.369870 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87efca8-73d0-4b26-84a0-b4909e60047a None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:31.370789 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87efca8-73d0-4b26-84a0-b4909e60047a None admin] PATH_INFO: `/v3/users/ab58de7f803c4cd29f61931966b168d5` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:31.372684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87efca8-73d0-4b26-84a0-b4909e60047a None admin] RBAC: Authorizing `identity:get_user(user_id=ab58de7f803c4cd29f61931966b168d5)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:31.377230 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87efca8-73d0-4b26-84a0-b4909e60047a None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:31.379168 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 78/153] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:23:31 2022] GET /identity/v3/users/ab58de7f803c4cd29f61931966b168d5 => generated 275 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:23:31.383212 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87efca8-73d0-4b26-84a0-b4909e60047a None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:31.429688 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:31.524753 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9532746-521e-4ea4-adf7-5c771d3d8ff9 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:31.607383 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87efca8-73d0-4b26-84a0-b4909e60047a None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:31.611140 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32345077-43b4-4cb9-909e-4fb465bd9b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:31.611790 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32345077-43b4-4cb9-909e-4fb465bd9b5b None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:31.612071 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32345077-43b4-4cb9-909e-4fb465bd9b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:31.612379 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32345077-43b4-4cb9-909e-4fb465bd9b5b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:31.634237 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32345077-43b4-4cb9-909e-4fb465bd9b5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:23:31.640867 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-32345077-43b4-4cb9-909e-4fb465bd9b5b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:23:31.640867 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:23:31.640867 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:23:31.640867 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:23:31.640867 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:23:31.640867 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:23:31.640867 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:23:31.640867 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:23:31.640867 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:23:31.640867 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:23:31.640867 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:23:31.640867 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:23:31.640867 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:23:31.640867 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:23:31.640867 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:23:31.640867 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:23:31.642120 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:23:31.642120 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:23:31.642120 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:23:31.642120 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:23:31.642120 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:23:31.642120 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:23:31.642120 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:23:31.642120 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:23:31.642120 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:23:31.642120 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:23:31.642120 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:23:31.642120 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:23:31.642120 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:23:31.642120 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:23:31.642120 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:23:31.642120 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:23:31.643104 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:23:31.643104 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:23:31.643104 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:23:31.643104 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:23:31.643104 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:23:31.643104 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:23:31.643104 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:23:31.643104 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:23:31.643104 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:23:31.643104 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:23:31.643104 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:23:31.643104 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:23:31.643104 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 76/154] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:23:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:23:31.645832 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a6b0f1-65a5-436c-9cb1-ea5b21e28364 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:31.646148 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a6b0f1-65a5-436c-9cb1-ea5b21e28364 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:31.648749 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a6b0f1-65a5-436c-9cb1-ea5b21e28364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:31.681622 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a73b4f9-5762-43c0-969b-921cdaaafe7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:31.682178 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a73b4f9-5762-43c0-969b-921cdaaafe7b None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:31.682466 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a73b4f9-5762-43c0-969b-921cdaaafe7b None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:31.682742 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a73b4f9-5762-43c0-969b-921cdaaafe7b None admin] PATH_INFO: `/v3/users/2c997665052f41d1b074897ce9efcc3d` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:31.684891 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a73b4f9-5762-43c0-969b-921cdaaafe7b None admin] RBAC: Authorizing `identity:get_user(user_id=2c997665052f41d1b074897ce9efcc3d)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:31.686364 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a73b4f9-5762-43c0-969b-921cdaaafe7b None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:31.688313 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 79/155] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:23:31 2022] GET /identity/v3/users/2c997665052f41d1b074897ce9efcc3d => generated 289 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:31.692792 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a73b4f9-5762-43c0-969b-921cdaaafe7b None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:31.694164 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-40a6b0f1-65a5-436c-9cb1-ea5b21e28364 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:31.695777 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-40a6b0f1-65a5-436c-9cb1-ea5b21e28364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:31.698344 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:31.820185 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 77/156] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:31.821463 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a6b0f1-65a5-436c-9cb1-ea5b21e28364 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:31.866416 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:31.880423 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a73b4f9-5762-43c0-969b-921cdaaafe7b None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:31.931838 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6928be-8390-4264-8b20-4f3e1a0eb221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:31.932375 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6928be-8390-4264-8b20-4f3e1a0eb221 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:31.932694 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6928be-8390-4264-8b20-4f3e1a0eb221 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:31.932969 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6928be-8390-4264-8b20-4f3e1a0eb221 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:31.991489 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a6b0f1-65a5-436c-9cb1-ea5b21e28364 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:32.040538 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6928be-8390-4264-8b20-4f3e1a0eb221 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:32.040538 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6928be-8390-4264-8b20-4f3e1a0eb221 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:32.065140 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fc2523-4ba8-4147-906c-1f8ee539f0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:32.066484 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fc2523-4ba8-4147-906c-1f8ee539f0b1 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:32.066987 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fc2523-4ba8-4147-906c-1f8ee539f0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:32.067453 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fc2523-4ba8-4147-906c-1f8ee539f0b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:32.068337 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fc2523-4ba8-4147-906c-1f8ee539f0b1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:32.068886 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fc2523-4ba8-4147-906c-1f8ee539f0b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:32.070471 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fc2523-4ba8-4147-906c-1f8ee539f0b1 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:32.076348 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:32.084175 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 78/157] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Aug 24 05:23:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:32.085270 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fc2523-4ba8-4147-906c-1f8ee539f0b1 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:32.200794 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:32.214876 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 80/158] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:23:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:23:32.218027 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6928be-8390-4264-8b20-4f3e1a0eb221 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:32.316719 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fc2523-4ba8-4147-906c-1f8ee539f0b1 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:32.327500 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6928be-8390-4264-8b20-4f3e1a0eb221 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:32.363033 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76021a4-02a0-4228-a776-559e927c3523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:32.363668 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76021a4-02a0-4228-a776-559e927c3523 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:32.363945 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76021a4-02a0-4228-a776-559e927c3523 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:32.364215 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76021a4-02a0-4228-a776-559e927c3523 None admin] PATH_INFO: `/v3/projects/4c7185cb4791495281d3a16293793beb` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:32.365484 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76021a4-02a0-4228-a776-559e927c3523 None admin] RBAC: Authorizing `identity:get_project(project_id=4c7185cb4791495281d3a16293793beb)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:32.366919 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76021a4-02a0-4228-a776-559e927c3523 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:32.370634 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 79/159] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:23:32 2022] GET /identity/v3/projects/4c7185cb4791495281d3a16293793beb => generated 298 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:32.371419 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76021a4-02a0-4228-a776-559e927c3523 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:32.397557 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e24851-77ae-4930-8a6f-2444da9f77c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:32.398216 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e24851-77ae-4930-8a6f-2444da9f77c6 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:32.398497 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e24851-77ae-4930-8a6f-2444da9f77c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:32.398767 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e24851-77ae-4930-8a6f-2444da9f77c6 None admin] PATH_INFO: `/v3/users/ab58de7f803c4cd29f61931966b168d5` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:32.399941 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e24851-77ae-4930-8a6f-2444da9f77c6 None admin] RBAC: Authorizing `identity:get_user(user_id=ab58de7f803c4cd29f61931966b168d5)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:32.401414 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e24851-77ae-4930-8a6f-2444da9f77c6 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:32.403184 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 81/160] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:23:32 2022] GET /identity/v3/users/ab58de7f803c4cd29f61931966b168d5 => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:32.404179 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671bc26a-7d1d-40cc-8803-dd245038c5b4 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:32.404489 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671bc26a-7d1d-40cc-8803-dd245038c5b4 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:32.404767 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671bc26a-7d1d-40cc-8803-dd245038c5b4 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:32.406364 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 82/161] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:32.413642 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671bc26a-7d1d-40cc-8803-dd245038c5b4 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:32.444944 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:32.482563 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76021a4-02a0-4228-a776-559e927c3523 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:32.521579 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd909edb-aaad-4dbe-bc97-672a059ac496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:32.522221 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd909edb-aaad-4dbe-bc97-672a059ac496 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:32.522499 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd909edb-aaad-4dbe-bc97-672a059ac496 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:32.522772 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd909edb-aaad-4dbe-bc97-672a059ac496 None admin] PATH_INFO: `/v3/projects/4c7185cb4791495281d3a16293793beb` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:32.525078 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd909edb-aaad-4dbe-bc97-672a059ac496 None admin] RBAC: Authorizing `identity:get_project(project_id=4c7185cb4791495281d3a16293793beb)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:32.526539 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd909edb-aaad-4dbe-bc97-672a059ac496 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:32.528357 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 80/162] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:23:32 2022] GET /identity/v3/projects/4c7185cb4791495281d3a16293793beb => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:32.528944 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd909edb-aaad-4dbe-bc97-672a059ac496 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:32.576755 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:32.639211 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671bc26a-7d1d-40cc-8803-dd245038c5b4 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:32.677705 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b0ba9a-c9b4-486e-b9d7-5d1aa6c20757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:32.678386 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b0ba9a-c9b4-486e-b9d7-5d1aa6c20757 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:32.678670 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b0ba9a-c9b4-486e-b9d7-5d1aa6c20757 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:32.679081 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b0ba9a-c9b4-486e-b9d7-5d1aa6c20757 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:32.690989 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b0ba9a-c9b4-486e-b9d7-5d1aa6c20757 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:23:32.696703 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-c2b0ba9a-c9b4-486e-b9d7-5d1aa6c20757 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:23:32.696703 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:23:32.696703 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:23:32.696703 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:23:32.696703 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:23:32.696703 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:23:32.696703 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:23:32.696703 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:23:32.696703 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:23:32.696703 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:23:32.696703 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:23:32.696703 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:23:32.696703 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:23:32.696703 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:23:32.696703 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:23:32.696703 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:23:32.697597 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:23:32.697597 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:23:32.697597 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:23:32.697597 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:23:32.697597 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:23:32.697597 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:23:32.697597 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:23:32.697597 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:23:32.697597 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:23:32.697597 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:23:32.697597 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:23:32.697597 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:23:32.697597 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:23:32.697597 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:23:32.697597 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:23:32.697597 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:23:32.698453 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:23:32.698453 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:23:32.698453 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:23:32.698453 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:23:32.698453 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:23:32.698453 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:23:32.698453 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:23:32.698453 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:23:32.698453 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:23:32.698453 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:23:32.698453 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:23:32.698453 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:23:32.698453 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 83/163] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Aug 24 05:23:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:23:32.699081 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b0ba9a-c9b4-486e-b9d7-5d1aa6c20757 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:32.715775 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd909edb-aaad-4dbe-bc97-672a059ac496 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:32.786914 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c374c17-aa9e-4667-820a-7aed64562466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:32.787591 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c374c17-aa9e-4667-820a-7aed64562466 None admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:32.787870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c374c17-aa9e-4667-820a-7aed64562466 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:32.788142 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c374c17-aa9e-4667-820a-7aed64562466 None admin] PATH_INFO: `/v3/projects/4c7185cb4791495281d3a16293793beb/users/2c997665052f41d1b074897ce9efcc3d/roles/49b52d60e4434f05870a37c49b2e6e6b` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:32.792699 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c374c17-aa9e-4667-820a-7aed64562466 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c7185cb4791495281d3a16293793beb, user_id=2c997665052f41d1b074897ce9efcc3d, role_id=49b52d60e4434f05870a37c49b2e6e6b)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:32.794176 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c374c17-aa9e-4667-820a-7aed64562466 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:32.800740 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:32.836545 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b0ba9a-c9b4-486e-b9d7-5d1aa6c20757 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:32.888191 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:32.899648 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 81/164] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Aug 24 05:23:32 2022] PUT /identity/v3/projects/4c7185cb4791495281d3a16293793beb/users/2c997665052f41d1b074897ce9efcc3d/roles/49b52d60e4434f05870a37c49b2e6e6b => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:23:32.901146 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4a6f2-7b09-4fa7-a53c-a9b23e408891 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:32.901432 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4a6f2-7b09-4fa7-a53c-a9b23e408891 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:32.901706 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4a6f2-7b09-4fa7-a53c-a9b23e408891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:32.929869 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc74562-65a5-4142-ae90-8ade61775acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:32.930498 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc74562-65a5-4142-ae90-8ade61775acf None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:32.930774 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc74562-65a5-4142-ae90-8ade61775acf None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:32.931023 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc74562-65a5-4142-ae90-8ade61775acf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:32.941476 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-ffd4a6f2-7b09-4fa7-a53c-a9b23e408891 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:32.942687 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-ffd4a6f2-7b09-4fa7-a53c-a9b23e408891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:33.076608 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 82/165] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:33.078057 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc74562-65a5-4142-ae90-8ade61775acf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:33.080051 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc74562-65a5-4142-ae90-8ade61775acf None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:33.084719 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd4a6f2-7b09-4fa7-a53c-a9b23e408891 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:33.212971 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 84/166] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:23:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:23:33.214618 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc74562-65a5-4142-ae90-8ade61775acf None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:33.243739 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:33.262899 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd4a6f2-7b09-4fa7-a53c-a9b23e408891 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:33.338720 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:33.373179 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0b996e-25ed-4d55-a539-a2a7dd3435a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:33.373791 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b996e-25ed-4d55-a539-a2a7dd3435a8 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:33.374072 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b996e-25ed-4d55-a539-a2a7dd3435a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:33.374341 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b996e-25ed-4d55-a539-a2a7dd3435a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:33.375124 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0b996e-25ed-4d55-a539-a2a7dd3435a8 None admin] RBAC: Adding query filter params (role.id=0cdb52ffa00449399b27dbfcd383f488, scope.project.id=4c7185cb4791495281d3a16293793beb, user.id=ab58de7f803c4cd29f61931966b168d5) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:33.375433 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0b996e-25ed-4d55-a539-a2a7dd3435a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:33.377605 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0b996e-25ed-4d55-a539-a2a7dd3435a8 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:33.394559 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 83/167] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:23:33 2022] GET /identity/v3/role_assignments?user.id=ab58de7f803c4cd29f61931966b168d5&scope.project.id=4c7185cb4791495281d3a16293793beb&role.id=0cdb52ffa00449399b27dbfcd383f488 => generated 608 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:33.395861 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eddf096-4398-485c-bcf2-cc1141ed7555 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:33.396113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eddf096-4398-485c-bcf2-cc1141ed7555 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:33.396387 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eddf096-4398-485c-bcf2-cc1141ed7555 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:33.397656 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 84/168] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:33.398364 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eddf096-4398-485c-bcf2-cc1141ed7555 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:33.427739 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc74562-65a5-4142-ae90-8ade61775acf None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:33.544423 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0acf16-af87-42c9-9148-ed399fe91e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:33.544423 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0acf16-af87-42c9-9148-ed399fe91e26 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:33.544423 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0acf16-af87-42c9-9148-ed399fe91e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:33.544423 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0acf16-af87-42c9-9148-ed399fe91e26 None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:33.544824 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0acf16-af87-42c9-9148-ed399fe91e26 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:33.544824 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0acf16-af87-42c9-9148-ed399fe91e26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:33.544824 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0acf16-af87-42c9-9148-ed399fe91e26 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:33.575250 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 85/169] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:23:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:33.576611 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680748e-9106-49c4-bf11-447de8b036ba None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:33.576898 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680748e-9106-49c4-bf11-447de8b036ba None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:33.577176 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680748e-9106-49c4-bf11-447de8b036ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:33.623745 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-2680748e-9106-49c4-bf11-447de8b036ba None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:33.625644 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-2680748e-9106-49c4-bf11-447de8b036ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:33.802846 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 86/170] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:33.805032 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680748e-9106-49c4-bf11-447de8b036ba None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:33.830022 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:33.853751 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eddf096-4398-485c-bcf2-cc1141ed7555 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:33.986679 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd20c3d-a9a8-4565-b6da-e22b6f4607e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:33.987133 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd20c3d-a9a8-4565-b6da-e22b6f4607e9 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:33.987419 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd20c3d-a9a8-4565-b6da-e22b6f4607e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:33.987701 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd20c3d-a9a8-4565-b6da-e22b6f4607e9 None admin] PATH_INFO: `/v3/projects/4c7185cb4791495281d3a16293793beb` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:33.988944 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd20c3d-a9a8-4565-b6da-e22b6f4607e9 None admin] RBAC: Authorizing `identity:get_project(project_id=4c7185cb4791495281d3a16293793beb)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:33.989047 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680748e-9106-49c4-bf11-447de8b036ba None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:33.990405 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd20c3d-a9a8-4565-b6da-e22b6f4607e9 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:33.992166 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 85/171] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:23:33 2022] GET /identity/v3/projects/4c7185cb4791495281d3a16293793beb => generated 298 bytes in 594 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:33.992887 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd20c3d-a9a8-4565-b6da-e22b6f4607e9 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:34.039541 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad5ab0d-ea7e-4197-a1ba-0c5b9692c9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:34.040192 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad5ab0d-ea7e-4197-a1ba-0c5b9692c9d9 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:34.040522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad5ab0d-ea7e-4197-a1ba-0c5b9692c9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:34.040814 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad5ab0d-ea7e-4197-a1ba-0c5b9692c9d9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:34.056426 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad5ab0d-ea7e-4197-a1ba-0c5b9692c9d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:23:34.059510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-cad5ab0d-ea7e-4197-a1ba-0c5b9692c9d9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:23:34.059510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:23:34.059510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:23:34.059510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:23:34.059510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:23:34.059510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:23:34.059510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:23:34.059510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:23:34.059510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:23:34.059510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:23:34.059510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:23:34.059510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:23:34.059510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:23:34.059510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:23:34.059510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:23:34.059510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:23:34.060457 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:23:34.060457 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:23:34.060457 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:23:34.060457 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:23:34.060457 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:23:34.060457 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:23:34.060457 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:23:34.060457 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:23:34.060457 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:23:34.060457 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:23:34.060457 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:23:34.060457 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:23:34.060457 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:23:34.060457 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:23:34.060457 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:23:34.060457 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:23:34.061240 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:23:34.061240 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:23:34.061240 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:23:34.061240 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:23:34.061240 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:23:34.061240 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:23:34.061240 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:23:34.061240 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:23:34.061240 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:23:34.061240 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:23:34.061240 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:23:34.061240 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:23:34.061934 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 87/172] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:23:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:23:34.062930 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c27ca8d-a1bb-4d39-83c4-65d75c228262 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:34.063267 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c27ca8d-a1bb-4d39-83c4-65d75c228262 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:34.063702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c27ca8d-a1bb-4d39-83c4-65d75c228262 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:34.066089 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 88/173] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:34.070905 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c27ca8d-a1bb-4d39-83c4-65d75c228262 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:34.097129 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:34.166227 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:34.172275 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd20c3d-a9a8-4565-b6da-e22b6f4607e9 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:34.210016 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c27ca8d-a1bb-4d39-83c4-65d75c228262 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:34.221257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5bffe6-b89b-4161-a7c6-2e87e7af2a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:34.221901 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5bffe6-b89b-4161-a7c6-2e87e7af2a8e None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:34.222187 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5bffe6-b89b-4161-a7c6-2e87e7af2a8e None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:34.222467 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5bffe6-b89b-4161-a7c6-2e87e7af2a8e None admin] PATH_INFO: `/v3/users/359799b7d001453b90d984a05215cce6` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:34.223731 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5bffe6-b89b-4161-a7c6-2e87e7af2a8e None admin] RBAC: Authorizing `identity:get_user(user_id=359799b7d001453b90d984a05215cce6)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:34.225209 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5bffe6-b89b-4161-a7c6-2e87e7af2a8e None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:34.227354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 86/174] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:23:33 2022] GET /identity/v3/users/359799b7d001453b90d984a05215cce6 => generated 247 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:34.228296 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5bffe6-b89b-4161-a7c6-2e87e7af2a8e None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:34.249395 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ccd5c-0859-4c45-834d-d17d5eedf407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:34.251661 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ccd5c-0859-4c45-834d-d17d5eedf407 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:34.252161 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ccd5c-0859-4c45-834d-d17d5eedf407 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:34.252782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ccd5c-0859-4c45-834d-d17d5eedf407 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:34.253620 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527ccd5c-0859-4c45-834d-d17d5eedf407 None admin] RBAC: Adding query filter params (role.id=87862cd6eaeb4d5fae089a801aac4f87, scope.project.id=4c7185cb4791495281d3a16293793beb, user.id=ab58de7f803c4cd29f61931966b168d5) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:34.254032 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527ccd5c-0859-4c45-834d-d17d5eedf407 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:34.255571 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527ccd5c-0859-4c45-834d-d17d5eedf407 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:34.264321 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 89/175] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:23:34 2022] GET /identity/v3/role_assignments?user.id=ab58de7f803c4cd29f61931966b168d5&scope.project.id=4c7185cb4791495281d3a16293793beb&role.id=87862cd6eaeb4d5fae089a801aac4f87 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:34.265667 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ffc41-ba8d-4994-bb50-7c29c84ab7ed None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:34.266214 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ffc41-ba8d-4994-bb50-7c29c84ab7ed None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:34.268017 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ffc41-ba8d-4994-bb50-7c29c84ab7ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:34.292032 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-c87ffc41-ba8d-4994-bb50-7c29c84ab7ed None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:34.293293 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-c87ffc41-ba8d-4994-bb50-7c29c84ab7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:34.376009 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5bffe6-b89b-4161-a7c6-2e87e7af2a8e None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:34.388202 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 90/176] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:34.390201 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b0a09-1094-4837-9a5a-01d4f15329cb None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:34.390945 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b0a09-1094-4837-9a5a-01d4f15329cb None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:34.392115 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b0a09-1094-4837-9a5a-01d4f15329cb None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:34.394442 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 91/177] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:34.395640 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2b0a09-1094-4837-9a5a-01d4f15329cb None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:34.417404 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:34.437602 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfe11c6-badb-4991-9519-7435771e7edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:34.438946 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfe11c6-badb-4991-9519-7435771e7edf None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:34.439235 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfe11c6-badb-4991-9519-7435771e7edf None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:34.439746 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfe11c6-badb-4991-9519-7435771e7edf None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:34.440936 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfe11c6-badb-4991-9519-7435771e7edf None admin] RBAC: Adding query filter params (name=member) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:34.441679 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfe11c6-badb-4991-9519-7435771e7edf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:34.443614 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfe11c6-badb-4991-9519-7435771e7edf None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:34.462744 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 87/178] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Aug 24 05:23:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:34.464070 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfe11c6-badb-4991-9519-7435771e7edf None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:34.486968 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:34.531867 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2b0a09-1094-4837-9a5a-01d4f15329cb None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:34.569855 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f857a78-ad7d-4523-92a1-d009e0e2dc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:34.571005 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f857a78-ad7d-4523-92a1-d009e0e2dc45 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:34.571005 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f857a78-ad7d-4523-92a1-d009e0e2dc45 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:34.571262 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f857a78-ad7d-4523-92a1-d009e0e2dc45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:34.586183 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfe11c6-badb-4991-9519-7435771e7edf None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:34.630556 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6a26f0-1218-44be-ae80-53da5f6a5c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:34.631566 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a26f0-1218-44be-ae80-53da5f6a5c39 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:34.632053 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a26f0-1218-44be-ae80-53da5f6a5c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:34.632571 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a26f0-1218-44be-ae80-53da5f6a5c39 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:34.645558 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6a26f0-1218-44be-ae80-53da5f6a5c39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:23:34.650604 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-3a6a26f0-1218-44be-ae80-53da5f6a5c39 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:23:34.650604 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:23:34.650604 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:23:34.650604 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:23:34.650604 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:23:34.650604 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:23:34.650604 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:23:34.650604 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:23:34.650604 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:23:34.650604 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:23:34.650604 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:23:34.650604 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:23:34.650604 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:23:34.650604 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:23:34.650604 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:23:34.650604 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:23:34.651485 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:23:34.651485 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:23:34.651485 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:23:34.651485 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:23:34.651485 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:23:34.651485 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:23:34.651485 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:23:34.651485 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:23:34.651485 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:23:34.651485 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:23:34.651485 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:23:34.651485 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:23:34.651485 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:23:34.651485 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:23:34.651485 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:23:34.651485 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:23:34.652302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:23:34.652302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:23:34.652302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:23:34.652302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:23:34.652302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:23:34.652302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:23:34.652302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:23:34.652302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:23:34.652302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:23:34.652302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:23:34.652302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:23:34.652302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:23:34.652302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 88/179] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Aug 24 05:23:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:23:34.652977 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b88348-e9b7-4c1d-8022-a6610ebd6df9 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:34.653826 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b88348-e9b7-4c1d-8022-a6610ebd6df9 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:34.653826 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b88348-e9b7-4c1d-8022-a6610ebd6df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:34.671001 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f857a78-ad7d-4523-92a1-d009e0e2dc45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:34.673342 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f857a78-ad7d-4523-92a1-d009e0e2dc45 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:34.673762 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-25b88348-e9b7-4c1d-8022-a6610ebd6df9 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:34.675902 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-25b88348-e9b7-4c1d-8022-a6610ebd6df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:34.724359 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:34.746080 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:34.752659 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 89/180] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:34.753732 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b88348-e9b7-4c1d-8022-a6610ebd6df9 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:34.802840 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 92/181] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:23:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:23:34.803461 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f857a78-ad7d-4523-92a1-d009e0e2dc45 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:34.906693 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b88348-e9b7-4c1d-8022-a6610ebd6df9 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:34.916088 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f857a78-ad7d-4523-92a1-d009e0e2dc45 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:34.953288 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893208cd-422a-4173-867b-a2e73eabf7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:34.954617 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893208cd-422a-4173-867b-a2e73eabf7c3 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:34.954972 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893208cd-422a-4173-867b-a2e73eabf7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:34.955319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893208cd-422a-4173-867b-a2e73eabf7c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:34.955990 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f359215d-62e9-4db5-a582-c35b871f191c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:34.956324 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893208cd-422a-4173-867b-a2e73eabf7c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:34.956732 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893208cd-422a-4173-867b-a2e73eabf7c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:34.957239 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359215d-62e9-4db5-a582-c35b871f191c None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:34.957696 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359215d-62e9-4db5-a582-c35b871f191c None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:34.958148 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359215d-62e9-4db5-a582-c35b871f191c None admin] PATH_INFO: `/v3/users/ab58de7f803c4cd29f61931966b168d5` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:34.958366 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893208cd-422a-4173-867b-a2e73eabf7c3 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:34.959683 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f359215d-62e9-4db5-a582-c35b871f191c None admin] RBAC: Authorizing `identity:get_user(user_id=ab58de7f803c4cd29f61931966b168d5)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:34.961907 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f359215d-62e9-4db5-a582-c35b871f191c None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:34.964504 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 90/182] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:23:34 2022] GET /identity/v3/users/ab58de7f803c4cd29f61931966b168d5 => generated 275 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:34.965821 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f359215d-62e9-4db5-a582-c35b871f191c None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:34.968007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 93/183] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:23:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:34.969256 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893208cd-422a-4173-867b-a2e73eabf7c3 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:34.998469 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:35.029537 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:35.077871 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893208cd-422a-4173-867b-a2e73eabf7c3 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:35.093904 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f359215d-62e9-4db5-a582-c35b871f191c None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:35.117339 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29f9f12-caf4-43e5-aaf0-71b762d35596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:35.118277 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f9f12-caf4-43e5-aaf0-71b762d35596 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:35.118563 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f9f12-caf4-43e5-aaf0-71b762d35596 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:35.118836 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f9f12-caf4-43e5-aaf0-71b762d35596 None admin] PATH_INFO: `/v3/projects/4c7185cb4791495281d3a16293793beb` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:35.120031 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29f9f12-caf4-43e5-aaf0-71b762d35596 None admin] RBAC: Authorizing `identity:get_project(project_id=4c7185cb4791495281d3a16293793beb)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:35.121523 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29f9f12-caf4-43e5-aaf0-71b762d35596 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:35.128241 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 94/184] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:23:34 2022] GET /identity/v3/projects/4c7185cb4791495281d3a16293793beb => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:35.129101 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29f9f12-caf4-43e5-aaf0-71b762d35596 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:35.133656 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4304cd-265e-4579-835f-6f7f2e93dc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:35.134290 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4304cd-265e-4579-835f-6f7f2e93dc00 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:35.134573 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4304cd-265e-4579-835f-6f7f2e93dc00 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:35.134845 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4304cd-265e-4579-835f-6f7f2e93dc00 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:35.146341 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4304cd-265e-4579-835f-6f7f2e93dc00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:23:35.151565 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-0f4304cd-265e-4579-835f-6f7f2e93dc00 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:23:35.151565 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:23:35.151565 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:23:35.151565 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:23:35.151565 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:23:35.151565 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:23:35.151565 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:23:35.151565 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:23:35.151565 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:23:35.151565 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:23:35.151565 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:23:35.151565 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:23:35.151565 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:23:35.151565 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:23:35.151565 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:23:35.151565 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:23:35.152487 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:23:35.152487 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:23:35.152487 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:23:35.152487 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:23:35.152487 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:23:35.152487 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:23:35.152487 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:23:35.152487 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:23:35.152487 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:23:35.152487 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:23:35.152487 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:23:35.152487 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:23:35.152487 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:23:35.152487 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:23:35.152487 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:23:35.152487 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:23:35.153284 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:23:35.153284 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:23:35.153284 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:23:35.153284 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:23:35.153284 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:23:35.153284 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:23:35.153284 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:23:35.153284 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:23:35.153284 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:23:35.153284 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:23:35.153284 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:23:35.153284 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:23:35.153284 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 91/185] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Aug 24 05:23:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:23:35.159286 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4304cd-265e-4579-835f-6f7f2e93dc00 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:35.237155 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:35.244480 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29f9f12-caf4-43e5-aaf0-71b762d35596 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:35.285208 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0487e06b-fdbc-4215-87ba-70986c205a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:35.286334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487e06b-fdbc-4215-87ba-70986c205a1a None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:35.287095 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487e06b-fdbc-4215-87ba-70986c205a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:35.287972 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487e06b-fdbc-4215-87ba-70986c205a1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:35.294678 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4304cd-265e-4579-835f-6f7f2e93dc00 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:35.304007 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:35.339794 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c06d485-e849-45c6-b512-289d420f22fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:35.340355 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c06d485-e849-45c6-b512-289d420f22fb None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:35.340946 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c06d485-e849-45c6-b512-289d420f22fb None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:35.341114 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c06d485-e849-45c6-b512-289d420f22fb None admin] PATH_INFO: `/v3/users/2cfe991c2c894beea1379ba2c7dac1a9` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:35.357624 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c06d485-e849-45c6-b512-289d420f22fb None admin] RBAC: Authorizing `identity:get_user(user_id=2cfe991c2c894beea1379ba2c7dac1a9)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:35.359107 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c06d485-e849-45c6-b512-289d420f22fb None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:35.365515 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 92/186] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:23:35 2022] GET /identity/v3/users/2cfe991c2c894beea1379ba2c7dac1a9 => generated 283 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:35.365515 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c06d485-e849-45c6-b512-289d420f22fb None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:35.367516 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0487e06b-fdbc-4215-87ba-70986c205a1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:35.369085 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0487e06b-fdbc-4215-87ba-70986c205a1a None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:35.478235 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:35.487900 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 95/187] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:23:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:23:35.491592 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0487e06b-fdbc-4215-87ba-70986c205a1a None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:35.495030 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c06d485-e849-45c6-b512-289d420f22fb None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:35.536874 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea27b859-d4d5-4a09-919e-6386382a092f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:35.537573 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27b859-d4d5-4a09-919e-6386382a092f None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:35.537864 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27b859-d4d5-4a09-919e-6386382a092f None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:35.538136 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27b859-d4d5-4a09-919e-6386382a092f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:35.539317 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea27b859-d4d5-4a09-919e-6386382a092f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:35.540792 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea27b859-d4d5-4a09-919e-6386382a092f None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:35.542489 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 93/188] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:23:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:35.545389 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea27b859-d4d5-4a09-919e-6386382a092f None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:35.584596 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:35.631330 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0487e06b-fdbc-4215-87ba-70986c205a1a None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:35.674199 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ca599f-c232-4748-bba8-5e98e6645938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:35.674812 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca599f-c232-4748-bba8-5e98e6645938 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:35.675096 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca599f-c232-4748-bba8-5e98e6645938 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:35.675371 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca599f-c232-4748-bba8-5e98e6645938 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:35.676144 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ca599f-c232-4748-bba8-5e98e6645938 None admin] RBAC: Adding query filter params (role.id=e013ad4d436f4cb0a9bf7debb7d13dc7, scope.project.id=4c7185cb4791495281d3a16293793beb, user.id=359799b7d001453b90d984a05215cce6) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:35.676453 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ca599f-c232-4748-bba8-5e98e6645938 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:35.677907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ca599f-c232-4748-bba8-5e98e6645938 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:35.685595 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 96/189] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:23:35 2022] GET /identity/v3/role_assignments?user.id=359799b7d001453b90d984a05215cce6&scope.project.id=4c7185cb4791495281d3a16293793beb&role.id=e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:35.686929 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ca599f-c232-4748-bba8-5e98e6645938 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:35.689235 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea27b859-d4d5-4a09-919e-6386382a092f None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:35.734150 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd55410-e36a-4f08-be10-8e237fe398fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:35.734825 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd55410-e36a-4f08-be10-8e237fe398fd None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:35.735134 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd55410-e36a-4f08-be10-8e237fe398fd None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:35.735467 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd55410-e36a-4f08-be10-8e237fe398fd None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:35.736169 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd55410-e36a-4f08-be10-8e237fe398fd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:35.736598 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd55410-e36a-4f08-be10-8e237fe398fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:35.738038 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd55410-e36a-4f08-be10-8e237fe398fd None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:35.750150 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 94/190] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:23:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:35.755899 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3823f2-4d69-4f73-9c48-7627ab0bf15e None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:35.755899 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3823f2-4d69-4f73-9c48-7627ab0bf15e None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:35.755899 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3823f2-4d69-4f73-9c48-7627ab0bf15e None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:35.755899 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 95/191] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:35.755899 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3823f2-4d69-4f73-9c48-7627ab0bf15e None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:35.780796 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:35.836147 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ca599f-c232-4748-bba8-5e98e6645938 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:35.882444 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:35.887984 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425168f-886a-4c6d-bea1-9cd8efa0f151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:35.888846 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425168f-886a-4c6d-bea1-9cd8efa0f151 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:35.889280 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425168f-886a-4c6d-bea1-9cd8efa0f151 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:35.896005 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425168f-886a-4c6d-bea1-9cd8efa0f151 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:35.896944 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3823f2-4d69-4f73-9c48-7627ab0bf15e None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:35.949387 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ee4b0-f8f9-47cf-85f9-4925f3d9d85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:35.950410 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ee4b0-f8f9-47cf-85f9-4925f3d9d85a None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:35.950884 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ee4b0-f8f9-47cf-85f9-4925f3d9d85a None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:35.951344 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ee4b0-f8f9-47cf-85f9-4925f3d9d85a None admin] PATH_INFO: `/v3/projects/f2dcd698555f4de580cb7ea6b098dbd7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:35.953254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852ee4b0-f8f9-47cf-85f9-4925f3d9d85a None admin] RBAC: Authorizing `identity:get_project(project_id=f2dcd698555f4de580cb7ea6b098dbd7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:35.955817 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852ee4b0-f8f9-47cf-85f9-4925f3d9d85a None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:35.958742 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 96/192] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:23:35 2022] GET /identity/v3/projects/f2dcd698555f4de580cb7ea6b098dbd7 => generated 312 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:35.959989 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ee4b0-f8f9-47cf-85f9-4925f3d9d85a None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:35.976783 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a425168f-886a-4c6d-bea1-9cd8efa0f151 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:35.978256 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a425168f-886a-4c6d-bea1-9cd8efa0f151 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:36.046162 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:36.074352 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ee4b0-f8f9-47cf-85f9-4925f3d9d85a None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:36.101871 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 97/193] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:23:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:23:36.101871 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425168f-886a-4c6d-bea1-9cd8efa0f151 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:36.113638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a7f3be-1663-4f72-9f25-7d4c296c0fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:36.115741 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a7f3be-1663-4f72-9f25-7d4c296c0fc3 None admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:36.116027 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a7f3be-1663-4f72-9f25-7d4c296c0fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:36.116306 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a7f3be-1663-4f72-9f25-7d4c296c0fc3 None admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:36.116999 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a7f3be-1663-4f72-9f25-7d4c296c0fc3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:36.118462 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a7f3be-1663-4f72-9f25-7d4c296c0fc3 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:36.222795 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425168f-886a-4c6d-bea1-9cd8efa0f151 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:36.236124 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 97/194] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Aug 24 05:23:35 2022] POST /identity/v3/users => generated 297 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:23:36.236124 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17bf7b-ee6b-40a2-af65-046370bdb8c2 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:36.236124 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17bf7b-ee6b-40a2-af65-046370bdb8c2 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:36.236124 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17bf7b-ee6b-40a2-af65-046370bdb8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:36.259676 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-dc17bf7b-ee6b-40a2-af65-046370bdb8c2 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:36.260957 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-dc17bf7b-ee6b-40a2-af65-046370bdb8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:36.277190 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e3fdcf-e958-4a97-8a02-08206a59aaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:36.278286 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e3fdcf-e958-4a97-8a02-08206a59aaf2 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:36.278763 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e3fdcf-e958-4a97-8a02-08206a59aaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:36.279224 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e3fdcf-e958-4a97-8a02-08206a59aaf2 None admin] PATH_INFO: `/v3/users/2cfe991c2c894beea1379ba2c7dac1a9` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:36.280089 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:36.282101 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e3fdcf-e958-4a97-8a02-08206a59aaf2 None admin] RBAC: Authorizing `identity:get_user(user_id=2cfe991c2c894beea1379ba2c7dac1a9)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:36.284640 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e3fdcf-e958-4a97-8a02-08206a59aaf2 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:36.287392 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 98/195] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:23:36 2022] GET /identity/v3/users/2cfe991c2c894beea1379ba2c7dac1a9 => generated 283 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:36.288016 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e3fdcf-e958-4a97-8a02-08206a59aaf2 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:36.326438 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:36.347180 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 98/196] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:36.348802 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc17bf7b-ee6b-40a2-af65-046370bdb8c2 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:36.436982 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e3fdcf-e958-4a97-8a02-08206a59aaf2 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:36.479472 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ae3952-afec-4912-85ae-f025a16b6687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:36.481565 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc17bf7b-ee6b-40a2-af65-046370bdb8c2 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:36.481680 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae3952-afec-4912-85ae-f025a16b6687 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:36.481957 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae3952-afec-4912-85ae-f025a16b6687 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:36.482290 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae3952-afec-4912-85ae-f025a16b6687 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:36.483089 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ae3952-afec-4912-85ae-f025a16b6687 None admin] RBAC: Adding query filter params (role.id=87862cd6eaeb4d5fae089a801aac4f87, scope.project.id=f2dcd698555f4de580cb7ea6b098dbd7, user.id=ab58de7f803c4cd29f61931966b168d5) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:36.483405 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ae3952-afec-4912-85ae-f025a16b6687 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:36.486038 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ae3952-afec-4912-85ae-f025a16b6687 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:36.494039 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 99/197] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:23:36 2022] GET /identity/v3/role_assignments?user.id=ab58de7f803c4cd29f61931966b168d5&scope.project.id=f2dcd698555f4de580cb7ea6b098dbd7&role.id=87862cd6eaeb4d5fae089a801aac4f87 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:36.497035 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ae3952-afec-4912-85ae-f025a16b6687 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:36.526599 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35afc6ff-382f-44b0-a880-dd4cb80ee85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:36.527181 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35afc6ff-382f-44b0-a880-dd4cb80ee85c None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:36.527467 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35afc6ff-382f-44b0-a880-dd4cb80ee85c None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:36.527743 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35afc6ff-382f-44b0-a880-dd4cb80ee85c None admin] PATH_INFO: `/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:36.530270 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35afc6ff-382f-44b0-a880-dd4cb80ee85c None admin] RBAC: Authorizing `identity:get_project(project_id=a7f9d22fd79e4b1fa16a2f60fc6faec7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:36.533504 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35afc6ff-382f-44b0-a880-dd4cb80ee85c None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:36.535970 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 99/198] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:23:36 2022] GET /identity/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:36.541936 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35afc6ff-382f-44b0-a880-dd4cb80ee85c None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:36.568680 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:36.646149 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:36.659467 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ae3952-afec-4912-85ae-f025a16b6687 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:36.684130 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35afc6ff-382f-44b0-a880-dd4cb80ee85c None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:36.705613 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0441e0f6-7abb-47f9-8917-b4ca3a6d8d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:36.706184 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0441e0f6-7abb-47f9-8917-b4ca3a6d8d38 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:36.706465 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0441e0f6-7abb-47f9-8917-b4ca3a6d8d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:36.706739 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0441e0f6-7abb-47f9-8917-b4ca3a6d8d38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:36.733312 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d77f8f-347d-46c7-b9bd-7904521e76e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:36.733939 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d77f8f-347d-46c7-b9bd-7904521e76e6 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:36.735563 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d77f8f-347d-46c7-b9bd-7904521e76e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:36.735850 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d77f8f-347d-46c7-b9bd-7904521e76e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:36.736767 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d77f8f-347d-46c7-b9bd-7904521e76e6 None admin] RBAC: Adding query filter params (role.id=e013ad4d436f4cb0a9bf7debb7d13dc7, scope.project.id=a7f9d22fd79e4b1fa16a2f60fc6faec7, user.id=2cfe991c2c894beea1379ba2c7dac1a9) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:36.737080 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d77f8f-347d-46c7-b9bd-7904521e76e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:36.738590 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d77f8f-347d-46c7-b9bd-7904521e76e6 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:36.746346 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 100/199] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:23:36 2022] GET /identity/v3/role_assignments?user.id=2cfe991c2c894beea1379ba2c7dac1a9&scope.project.id=a7f9d22fd79e4b1fa16a2f60fc6faec7&role.id=e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:23:36.747340 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d77f8f-347d-46c7-b9bd-7904521e76e6 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:36.825305 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0441e0f6-7abb-47f9-8917-b4ca3a6d8d38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:36.827530 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0441e0f6-7abb-47f9-8917-b4ca3a6d8d38 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:36.873594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:36.906105 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d77f8f-347d-46c7-b9bd-7904521e76e6 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:36.950324 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdabe41-6624-42e5-bd66-78ef53b1d858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:36.951249 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdabe41-6624-42e5-bd66-78ef53b1d858 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:36.951682 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdabe41-6624-42e5-bd66-78ef53b1d858 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:36.952206 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdabe41-6624-42e5-bd66-78ef53b1d858 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:36.965199 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdabe41-6624-42e5-bd66-78ef53b1d858 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:23:36.970574 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-afdabe41-6624-42e5-bd66-78ef53b1d858 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:23:36.970574 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:23:36.970574 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:23:36.970574 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:23:36.970574 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:23:36.970574 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:23:36.970574 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:23:36.970574 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:23:36.970574 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:23:36.970574 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:23:36.970574 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:23:36.970574 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:23:36.970574 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:23:36.970574 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:23:36.970574 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:23:36.970574 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:23:36.971650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:23:36.971650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:23:36.971650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:23:36.971650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:23:36.971650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:23:36.971650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:23:36.971650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:23:36.971650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:23:36.971650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:23:36.971650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:23:36.971650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:23:36.971650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:23:36.971650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:23:36.971650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:23:36.971650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:23:36.971650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:23:36.972665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:23:36.972665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:23:36.972665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:23:36.972665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:23:36.972665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:23:36.972665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:23:36.972665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:23:36.972665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:23:36.972665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:23:36.972665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:23:36.972665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:23:36.972665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:23:36.974703 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 101/200] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:23:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:23:36.980070 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:36.981844 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdabe41-6624-42e5-bd66-78ef53b1d858 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:36.991161 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 100/201] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:23:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:23:37.002477 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0441e0f6-7abb-47f9-8917-b4ca3a6d8d38 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:37.098845 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdabe41-6624-42e5-bd66-78ef53b1d858 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:37.137490 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93917ace-230b-44c8-90a1-d475053525ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:37.138351 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93917ace-230b-44c8-90a1-d475053525ad None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:37.138810 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93917ace-230b-44c8-90a1-d475053525ad None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:37.139235 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93917ace-230b-44c8-90a1-d475053525ad None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:37.140057 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93917ace-230b-44c8-90a1-d475053525ad None admin] RBAC: Adding query filter params (name=reader) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:37.140539 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93917ace-230b-44c8-90a1-d475053525ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:37.142231 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93917ace-230b-44c8-90a1-d475053525ad None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:37.151616 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 102/202] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Aug 24 05:23:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:37.158173 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93917ace-230b-44c8-90a1-d475053525ad None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:37.162205 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:37.248552 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0441e0f6-7abb-47f9-8917-b4ca3a6d8d38 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:37.287994 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93917ace-230b-44c8-90a1-d475053525ad None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:37.326915 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f549d9ee-43fc-4071-bb44-e9173020a005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:37.327817 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f549d9ee-43fc-4071-bb44-e9173020a005 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:37.328104 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f549d9ee-43fc-4071-bb44-e9173020a005 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:37.328375 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f549d9ee-43fc-4071-bb44-e9173020a005 None admin] PATH_INFO: `/v3/users/2c997665052f41d1b074897ce9efcc3d` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:37.330597 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f549d9ee-43fc-4071-bb44-e9173020a005 None admin] RBAC: Authorizing `identity:get_user(user_id=2c997665052f41d1b074897ce9efcc3d)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:37.332064 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f549d9ee-43fc-4071-bb44-e9173020a005 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:37.334070 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 103/203] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:23:37 2022] GET /identity/v3/users/2c997665052f41d1b074897ce9efcc3d => generated 289 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:23:37.339423 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f549d9ee-43fc-4071-bb44-e9173020a005 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:37.345653 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f0fc2-e28c-4e34-8b9e-fc102acef530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:37.346312 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f0fc2-e28c-4e34-8b9e-fc102acef530 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:37.346590 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f0fc2-e28c-4e34-8b9e-fc102acef530 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:37.346860 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f0fc2-e28c-4e34-8b9e-fc102acef530 None admin] PATH_INFO: `/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:37.353133 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f0fc2-e28c-4e34-8b9e-fc102acef530 None admin] RBAC: Authorizing `identity:get_project(project_id=a7f9d22fd79e4b1fa16a2f60fc6faec7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:37.354616 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f0fc2-e28c-4e34-8b9e-fc102acef530 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:37.356586 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 101/204] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:23:37 2022] GET /identity/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7 => generated 302 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:37.362773 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f0fc2-e28c-4e34-8b9e-fc102acef530 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:37.436848 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:37.453581 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f549d9ee-43fc-4071-bb44-e9173020a005 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:37.523117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786cd974-1fed-4199-9956-4b574e9f6f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:37.523670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786cd974-1fed-4199-9956-4b574e9f6f4d None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:37.524010 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786cd974-1fed-4199-9956-4b574e9f6f4d None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:37.524295 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786cd974-1fed-4199-9956-4b574e9f6f4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:37.526309 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:37.573222 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f0fc2-e28c-4e34-8b9e-fc102acef530 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:37.611893 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6757cd2-cc17-4866-9a37-df3deee97245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:37.612917 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6757cd2-cc17-4866-9a37-df3deee97245 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:37.613370 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6757cd2-cc17-4866-9a37-df3deee97245 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:37.613926 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6757cd2-cc17-4866-9a37-df3deee97245 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:37.615147 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6757cd2-cc17-4866-9a37-df3deee97245 None admin] RBAC: Adding query filter params (role.id=0cdb52ffa00449399b27dbfcd383f488, scope.project.id=a7f9d22fd79e4b1fa16a2f60fc6faec7, user.id=2cfe991c2c894beea1379ba2c7dac1a9) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:37.615613 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6757cd2-cc17-4866-9a37-df3deee97245 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:37.617285 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6757cd2-cc17-4866-9a37-df3deee97245 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:37.624955 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 102/205] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:23:37 2022] GET /identity/v3/role_assignments?user.id=2cfe991c2c894beea1379ba2c7dac1a9&scope.project.id=a7f9d22fd79e4b1fa16a2f60fc6faec7&role.id=0cdb52ffa00449399b27dbfcd383f488 => generated 259 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:37.685938 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786cd974-1fed-4199-9956-4b574e9f6f4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:37.687502 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786cd974-1fed-4199-9956-4b574e9f6f4d None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:37.781257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:37.829821 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 104/206] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:23:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:23:37.836419 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6757cd2-cc17-4866-9a37-df3deee97245 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:37.960912 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6757cd2-cc17-4866-9a37-df3deee97245 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:37.984985 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:38.000097 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af96a0b-4e7c-4b64-ba5d-8ec57cf89d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:38.000833 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af96a0b-4e7c-4b64-ba5d-8ec57cf89d20 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:38.001116 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af96a0b-4e7c-4b64-ba5d-8ec57cf89d20 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:38.001393 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af96a0b-4e7c-4b64-ba5d-8ec57cf89d20 None admin] PATH_INFO: `/v3/projects/4c7185cb4791495281d3a16293793beb` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:38.002683 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af96a0b-4e7c-4b64-ba5d-8ec57cf89d20 None admin] RBAC: Authorizing `identity:get_project(project_id=4c7185cb4791495281d3a16293793beb)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:38.004174 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af96a0b-4e7c-4b64-ba5d-8ec57cf89d20 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:38.006326 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 103/207] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:23:37 2022] GET /identity/v3/projects/4c7185cb4791495281d3a16293793beb => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:38.017252 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786cd974-1fed-4199-9956-4b574e9f6f4d None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:38.171049 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786cd974-1fed-4199-9956-4b574e9f6f4d None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:38.231120 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c9798-4ce4-4dec-8546-cb0c9572e916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:38.232191 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c9798-4ce4-4dec-8546-cb0c9572e916 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:38.232751 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c9798-4ce4-4dec-8546-cb0c9572e916 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:38.233272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c9798-4ce4-4dec-8546-cb0c9572e916 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:38.234675 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86c9798-4ce4-4dec-8546-cb0c9572e916 None admin] RBAC: Adding query filter params (role.id=49b52d60e4434f05870a37c49b2e6e6b, scope.project.id=4c7185cb4791495281d3a16293793beb, user.id=2c997665052f41d1b074897ce9efcc3d) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:38.235998 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86c9798-4ce4-4dec-8546-cb0c9572e916 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:38.238839 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86c9798-4ce4-4dec-8546-cb0c9572e916 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:38.253659 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 105/208] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:23:38 2022] GET /identity/v3/role_assignments?user.id=2c997665052f41d1b074897ce9efcc3d&scope.project.id=4c7185cb4791495281d3a16293793beb&role.id=49b52d60e4434f05870a37c49b2e6e6b => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:38.878143 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f122409-1b3d-4c5a-af19-529ed573eea0 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:38.879067 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f122409-1b3d-4c5a-af19-529ed573eea0 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:38.879524 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f122409-1b3d-4c5a-af19-529ed573eea0 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:38.881139 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 104/209] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:23:38.886968 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c27448-e3d3-49e7-ab07-04816f6e3ed0 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:38.887426 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c27448-e3d3-49e7-ab07-04816f6e3ed0 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:38.887637 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c27448-e3d3-49e7-ab07-04816f6e3ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:38.908470 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-76c27448-e3d3-49e7-ab07-04816f6e3ed0 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:38.909733 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-76c27448-e3d3-49e7-ab07-04816f6e3ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:38.933953 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4802c0-a8ce-4c12-aae2-2faf7fa2b143 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:38.934240 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4802c0-a8ce-4c12-aae2-2faf7fa2b143 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:38.934521 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4802c0-a8ce-4c12-aae2-2faf7fa2b143 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:38.936237 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 105/210] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:23:38.942583 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63137a4-5a56-45cf-8a4e-4d7dd2db4af8 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:38.943069 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63137a4-5a56-45cf-8a4e-4d7dd2db4af8 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:38.943565 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63137a4-5a56-45cf-8a4e-4d7dd2db4af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:38.945020 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:38.962653 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-a63137a4-5a56-45cf-8a4e-4d7dd2db4af8 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:38.964144 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-a63137a4-5a56-45cf-8a4e-4d7dd2db4af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:38.994904 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 106/211] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:39.003018 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c169a3-562a-467e-ad4e-c42ed04219e2 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:39.003310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c169a3-562a-467e-ad4e-c42ed04219e2 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:39.003591 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c169a3-562a-467e-ad4e-c42ed04219e2 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:39.005238 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 107/212] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:39.010817 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd797e-b354-44cc-aa67-6f95ff2cf337 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:39.011089 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd797e-b354-44cc-aa67-6f95ff2cf337 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:39.011377 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd797e-b354-44cc-aa67-6f95ff2cf337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:39.029620 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-98fd797e-b354-44cc-aa67-6f95ff2cf337 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:39.030911 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-98fd797e-b354-44cc-aa67-6f95ff2cf337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:39.048280 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 106/213] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:23:39.100179 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 108/214] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:23:39.266316 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63137a4-5a56-45cf-8a4e-4d7dd2db4af8 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:39.335609 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fd797e-b354-44cc-aa67-6f95ff2cf337 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:39.372192 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63137a4-5a56-45cf-8a4e-4d7dd2db4af8 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:39.394289 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:39.410220 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92608739-b55f-46ca-8d91-8e25e0506d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:39.411135 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92608739-b55f-46ca-8d91-8e25e0506d86 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:39.411418 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92608739-b55f-46ca-8d91-8e25e0506d86 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:39.411939 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92608739-b55f-46ca-8d91-8e25e0506d86 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:39.425365 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92608739-b55f-46ca-8d91-8e25e0506d86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:23:39.430954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-92608739-b55f-46ca-8d91-8e25e0506d86 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:23:39.430954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:23:39.430954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:23:39.430954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:23:39.430954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:23:39.430954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:23:39.430954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:23:39.430954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:23:39.430954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:23:39.430954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:23:39.430954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:23:39.430954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:23:39.430954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:23:39.430954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:23:39.430954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:23:39.430954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:23:39.431823 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:23:39.431823 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:23:39.431823 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:23:39.431823 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:23:39.431823 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:23:39.431823 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:23:39.431823 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:23:39.431823 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:23:39.431823 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:23:39.431823 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:23:39.431823 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:23:39.431823 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:23:39.431823 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:23:39.431823 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:23:39.431823 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:23:39.431823 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:23:39.437759 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:23:39.437759 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:23:39.437759 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:23:39.437759 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:23:39.437759 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:23:39.437759 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:23:39.437759 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:23:39.437759 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:23:39.437759 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:23:39.437759 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:23:39.437759 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:23:39.437759 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:23:39.437759 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 107/215] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:23:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:23:39.437759 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92608739-b55f-46ca-8d91-8e25e0506d86 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:39.439593 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:39.446522 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fd797e-b354-44cc-aa67-6f95ff2cf337 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:39.482925 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda8947-4a3c-4d98-a7e7-691d44de5a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:39.483563 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda8947-4a3c-4d98-a7e7-691d44de5a8b None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:39.483908 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda8947-4a3c-4d98-a7e7-691d44de5a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:39.484196 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda8947-4a3c-4d98-a7e7-691d44de5a8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:39.495486 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acda8947-4a3c-4d98-a7e7-691d44de5a8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:23:39.501239 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-acda8947-4a3c-4d98-a7e7-691d44de5a8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:23:39.501239 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:23:39.501239 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:23:39.501239 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:23:39.501239 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:23:39.501239 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:23:39.501239 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:23:39.501239 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:23:39.501239 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:23:39.501239 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:23:39.501239 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:23:39.501239 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:23:39.501239 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:23:39.501239 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:23:39.501239 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:23:39.501239 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:23:39.502119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:23:39.502119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:23:39.502119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:23:39.502119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:23:39.502119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:23:39.502119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:23:39.502119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:23:39.502119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:23:39.502119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:23:39.502119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:23:39.502119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:23:39.502119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:23:39.502119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:23:39.502119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:23:39.502119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:23:39.502119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:23:39.503255 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:23:39.503255 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:23:39.503255 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:23:39.503255 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:23:39.503255 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:23:39.503255 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:23:39.503255 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:23:39.503255 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:23:39.503255 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:23:39.503255 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:23:39.503255 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:23:39.503255 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:23:39.503255 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 109/216] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Aug 24 05:23:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:23:39.505265 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda8947-4a3c-4d98-a7e7-691d44de5a8b None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:39.551738 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92608739-b55f-46ca-8d91-8e25e0506d86 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:39.590295 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ae4037-a13c-4954-bdc2-186b9831df0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:39.591092 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae4037-a13c-4954-bdc2-186b9831df0b None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:39.591381 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae4037-a13c-4954-bdc2-186b9831df0b None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:39.591661 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae4037-a13c-4954-bdc2-186b9831df0b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:39.592833 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ae4037-a13c-4954-bdc2-186b9831df0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:39.594325 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ae4037-a13c-4954-bdc2-186b9831df0b None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:39.596434 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 108/217] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:23:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:39.597213 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ae4037-a13c-4954-bdc2-186b9831df0b None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:39.625802 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda8947-4a3c-4d98-a7e7-691d44de5a8b None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:39.653228 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:39.666380 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f55c8b-f28f-4878-8b81-e32caaf6715f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:39.667001 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f55c8b-f28f-4878-8b81-e32caaf6715f None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:39.667296 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f55c8b-f28f-4878-8b81-e32caaf6715f None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:39.667568 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f55c8b-f28f-4878-8b81-e32caaf6715f None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:39.668515 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f55c8b-f28f-4878-8b81-e32caaf6715f None admin] RBAC: Adding query filter params (name=member) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:39.668610 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f55c8b-f28f-4878-8b81-e32caaf6715f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:39.669995 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f55c8b-f28f-4878-8b81-e32caaf6715f None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:39.679243 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 110/218] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Aug 24 05:23:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:39.680053 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f55c8b-f28f-4878-8b81-e32caaf6715f None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:39.703365 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:39.711841 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ae4037-a13c-4954-bdc2-186b9831df0b None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:39.750480 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee9ad80-1fcf-487b-b03e-9c516e755633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:39.752087 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9ad80-1fcf-487b-b03e-9c516e755633 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:39.752372 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9ad80-1fcf-487b-b03e-9c516e755633 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:39.752679 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9ad80-1fcf-487b-b03e-9c516e755633 None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:39.753368 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee9ad80-1fcf-487b-b03e-9c516e755633 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:39.753843 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee9ad80-1fcf-487b-b03e-9c516e755633 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:39.756291 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee9ad80-1fcf-487b-b03e-9c516e755633 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:39.768443 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 109/219] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:23:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:39.768625 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee9ad80-1fcf-487b-b03e-9c516e755633 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:39.800331 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f55c8b-f28f-4878-8b81-e32caaf6715f None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:39.837347 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba42e8-8e08-4a1c-aabb-9f700c7a78f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:39.837958 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba42e8-8e08-4a1c-aabb-9f700c7a78f5 None admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:39.838242 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba42e8-8e08-4a1c-aabb-9f700c7a78f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:39.838516 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba42e8-8e08-4a1c-aabb-9f700c7a78f5 None admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:39.839160 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eba42e8-8e08-4a1c-aabb-9f700c7a78f5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:39.840661 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eba42e8-8e08-4a1c-aabb-9f700c7a78f5 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:39.880328 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee9ad80-1fcf-487b-b03e-9c516e755633 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:39.890585 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:39.917385 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a7293-b112-42ed-bb4a-d80c4af754d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:39.918026 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a7293-b112-42ed-bb4a-d80c4af754d7 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:39.918307 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a7293-b112-42ed-bb4a-d80c4af754d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:39.918582 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a7293-b112-42ed-bb4a-d80c4af754d7 None admin] PATH_INFO: `/v3/users/2117f4e51a7d4a7f9295f2f8c12e8713` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:39.934625 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9a7293-b112-42ed-bb4a-d80c4af754d7 None admin] RBAC: Authorizing `identity:get_user(user_id=2117f4e51a7d4a7f9295f2f8c12e8713)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:39.936072 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9a7293-b112-42ed-bb4a-d80c4af754d7 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:39.943686 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 110/220] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:23:39 2022] GET /identity/v3/users/2117f4e51a7d4a7f9295f2f8c12e8713 => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:39.943686 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a7293-b112-42ed-bb4a-d80c4af754d7 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:40.001450 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 111/221] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Aug 24 05:23:39 2022] POST /identity/v3/users => generated 297 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:23:40.002070 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba42e8-8e08-4a1c-aabb-9f700c7a78f5 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:40.068586 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a7293-b112-42ed-bb4a-d80c4af754d7 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:40.123874 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:40.124076 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e859290-4494-4376-aff2-7ad6b7c29d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:40.124594 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e859290-4494-4376-aff2-7ad6b7c29d06 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:40.124886 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e859290-4494-4376-aff2-7ad6b7c29d06 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:40.125170 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e859290-4494-4376-aff2-7ad6b7c29d06 None admin] PATH_INFO: `/v3/users/2cfe991c2c894beea1379ba2c7dac1a9` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:40.127204 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e859290-4494-4376-aff2-7ad6b7c29d06 None admin] RBAC: Authorizing `identity:get_user(user_id=2cfe991c2c894beea1379ba2c7dac1a9)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:40.128745 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e859290-4494-4376-aff2-7ad6b7c29d06 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:40.130611 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 111/222] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:23:39 2022] GET /identity/v3/users/2cfe991c2c894beea1379ba2c7dac1a9 => generated 283 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:40.131828 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3563ea-9479-4a4b-bcef-f69a56a95100 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:40.132124 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3563ea-9479-4a4b-bcef-f69a56a95100 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:40.132502 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3563ea-9479-4a4b-bcef-f69a56a95100 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:40.133811 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 112/223] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:40.137406 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3563ea-9479-4a4b-bcef-f69a56a95100 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:40.167250 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba42e8-8e08-4a1c-aabb-9f700c7a78f5 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:40.193066 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:40.211841 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1207d56-5632-489d-ab10-542ff0e0513c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:40.213111 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1207d56-5632-489d-ab10-542ff0e0513c None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:40.213396 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1207d56-5632-489d-ab10-542ff0e0513c None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:40.213671 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1207d56-5632-489d-ab10-542ff0e0513c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:40.283841 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3563ea-9479-4a4b-bcef-f69a56a95100 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:40.310790 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1207d56-5632-489d-ab10-542ff0e0513c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:40.312615 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1207d56-5632-489d-ab10-542ff0e0513c None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:40.339512 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de994f8f-d011-41b8-b45e-76dc060c3dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:40.340307 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de994f8f-d011-41b8-b45e-76dc060c3dde None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:40.340643 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de994f8f-d011-41b8-b45e-76dc060c3dde None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:40.340923 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de994f8f-d011-41b8-b45e-76dc060c3dde None admin] PATH_INFO: `/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:40.342323 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de994f8f-d011-41b8-b45e-76dc060c3dde None admin] RBAC: Authorizing `identity:get_project(project_id=a7f9d22fd79e4b1fa16a2f60fc6faec7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:40.343492 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de994f8f-d011-41b8-b45e-76dc060c3dde None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:40.345717 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 113/224] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:23:40 2022] GET /identity/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7 => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:40.347123 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab485b-eb86-4017-893c-bed95169df5f None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:40.347419 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab485b-eb86-4017-893c-bed95169df5f None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:40.347696 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab485b-eb86-4017-893c-bed95169df5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:40.369771 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-06ab485b-eb86-4017-893c-bed95169df5f None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:40.372307 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-06ab485b-eb86-4017-893c-bed95169df5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:40.399905 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:40.432850 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 112/225] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:23:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:23:40.433752 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1207d56-5632-489d-ab10-542ff0e0513c None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:40.447411 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 114/226] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:40.448104 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ab485b-eb86-4017-893c-bed95169df5f None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:40.484007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:40.555825 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ab485b-eb86-4017-893c-bed95169df5f None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:40.559745 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1207d56-5632-489d-ab10-542ff0e0513c None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:40.595260 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00698039-7dd7-4ce6-9279-26326c82a778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:40.596618 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00698039-7dd7-4ce6-9279-26326c82a778 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:40.597236 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00698039-7dd7-4ce6-9279-26326c82a778 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:40.597593 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00698039-7dd7-4ce6-9279-26326c82a778 None admin] PATH_INFO: `/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:40.599300 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00698039-7dd7-4ce6-9279-26326c82a778 None admin] RBAC: Authorizing `identity:get_project(project_id=a7f9d22fd79e4b1fa16a2f60fc6faec7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:40.601474 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00698039-7dd7-4ce6-9279-26326c82a778 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:40.603627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 115/227] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:23:40 2022] GET /identity/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:40.603737 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42c6e48-6f04-4197-875a-034823076e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:40.604629 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c6e48-6f04-4197-875a-034823076e0e None admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:40.604912 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c6e48-6f04-4197-875a-034823076e0e None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:40.605187 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c6e48-6f04-4197-875a-034823076e0e None admin] PATH_INFO: `/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7/users/2cfe991c2c894beea1379ba2c7dac1a9/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:40.609498 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00698039-7dd7-4ce6-9279-26326c82a778 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:40.609836 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42c6e48-6f04-4197-875a-034823076e0e None admin] RBAC: Authorizing `identity:create_grant(project_id=a7f9d22fd79e4b1fa16a2f60fc6faec7, user_id=2cfe991c2c894beea1379ba2c7dac1a9, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:40.612205 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42c6e48-6f04-4197-875a-034823076e0e None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:40.719845 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 113/228] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Aug 24 05:23:40 2022] PUT /identity/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7/users/2cfe991c2c894beea1379ba2c7dac1a9/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:23:40.727675 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00698039-7dd7-4ce6-9279-26326c82a778 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:40.730006 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:40.768428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca1f66f-8911-4861-b7a9-d1ccb5c2ed4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:40.769145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca1f66f-8911-4861-b7a9-d1ccb5c2ed4d None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:40.769426 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca1f66f-8911-4861-b7a9-d1ccb5c2ed4d None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:40.769701 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca1f66f-8911-4861-b7a9-d1ccb5c2ed4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:40.770485 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca1f66f-8911-4861-b7a9-d1ccb5c2ed4d None admin] RBAC: Adding query filter params (role.id=87862cd6eaeb4d5fae089a801aac4f87, scope.project.id=a7f9d22fd79e4b1fa16a2f60fc6faec7, user.id=2117f4e51a7d4a7f9295f2f8c12e8713) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:40.770813 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca1f66f-8911-4861-b7a9-d1ccb5c2ed4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:40.772296 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca1f66f-8911-4861-b7a9-d1ccb5c2ed4d None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:40.778744 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 116/229] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:23:40 2022] GET /identity/v3/role_assignments?user.id=2117f4e51a7d4a7f9295f2f8c12e8713&scope.project.id=a7f9d22fd79e4b1fa16a2f60fc6faec7&role.id=87862cd6eaeb4d5fae089a801aac4f87 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:40.798518 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42c6e48-6f04-4197-875a-034823076e0e None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:40.834088 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:40.934897 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42c6e48-6f04-4197-875a-034823076e0e None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:40.981388 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3721ebb-df66-4141-a07e-3881799b8c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:40.982094 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3721ebb-df66-4141-a07e-3881799b8c50 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:40.982384 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3721ebb-df66-4141-a07e-3881799b8c50 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:40.982661 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3721ebb-df66-4141-a07e-3881799b8c50 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:40.995037 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3721ebb-df66-4141-a07e-3881799b8c50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:23:40.999894 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-a3721ebb-df66-4141-a07e-3881799b8c50 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:23:40.999894 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:23:40.999894 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:23:40.999894 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:23:40.999894 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:23:40.999894 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:23:40.999894 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:23:40.999894 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:23:40.999894 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:23:40.999894 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:23:40.999894 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:23:40.999894 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:23:40.999894 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:23:40.999894 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:23:40.999894 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:23:40.999894 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:23:41.000857 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:23:41.000857 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:23:41.000857 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:23:41.000857 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:23:41.000857 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:23:41.000857 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:23:41.000857 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:23:41.000857 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:23:41.000857 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:23:41.000857 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:23:41.000857 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:23:41.000857 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:23:41.000857 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:23:41.000857 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:23:41.000857 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:23:41.000857 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:23:41.002078 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:23:41.002078 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:23:41.002078 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:23:41.002078 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:23:41.002078 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:23:41.002078 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:23:41.002078 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:23:41.002078 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:23:41.002078 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:23:41.002078 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:23:41.002078 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:23:41.002078 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:23:41.006573 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 114/230] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Aug 24 05:23:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:23:41.013587 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca1f66f-8911-4861-b7a9-d1ccb5c2ed4d None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:41.127285 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca1f66f-8911-4861-b7a9-d1ccb5c2ed4d None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:41.169854 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce80e221-da6a-49ec-b97f-9a8dc4ec82d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:41.170488 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80e221-da6a-49ec-b97f-9a8dc4ec82d0 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:41.170798 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80e221-da6a-49ec-b97f-9a8dc4ec82d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:41.171078 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80e221-da6a-49ec-b97f-9a8dc4ec82d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:41.171765 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce80e221-da6a-49ec-b97f-9a8dc4ec82d0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:41.172130 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce80e221-da6a-49ec-b97f-9a8dc4ec82d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:41.174105 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce80e221-da6a-49ec-b97f-9a8dc4ec82d0 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:41.190298 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 117/231] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:23:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:41.192237 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3721ebb-df66-4141-a07e-3881799b8c50 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:41.352230 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:41.366377 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3721ebb-df66-4141-a07e-3881799b8c50 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:41.412264 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582f1e7f-265a-4fac-9f8f-8fa8b993b7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:41.413490 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f1e7f-265a-4fac-9f8f-8fa8b993b7b3 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:41.413996 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f1e7f-265a-4fac-9f8f-8fa8b993b7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:41.418121 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f1e7f-265a-4fac-9f8f-8fa8b993b7b3 None admin] PATH_INFO: `/v3/users/2cfe991c2c894beea1379ba2c7dac1a9` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:41.419743 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582f1e7f-265a-4fac-9f8f-8fa8b993b7b3 None admin] RBAC: Authorizing `identity:get_user(user_id=2cfe991c2c894beea1379ba2c7dac1a9)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:41.421494 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582f1e7f-265a-4fac-9f8f-8fa8b993b7b3 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:41.424207 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 115/232] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:23:41 2022] GET /identity/v3/users/2cfe991c2c894beea1379ba2c7dac1a9 => generated 283 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:41.430153 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce80e221-da6a-49ec-b97f-9a8dc4ec82d0 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:41.483578 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:41.567719 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce80e221-da6a-49ec-b97f-9a8dc4ec82d0 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:41.612863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8bb626-f371-4f81-a366-a6c03020c1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:41.613523 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8bb626-f371-4f81-a366-a6c03020c1fc None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:41.613810 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8bb626-f371-4f81-a366-a6c03020c1fc None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:41.614093 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8bb626-f371-4f81-a366-a6c03020c1fc None admin] PATH_INFO: `/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:41.615315 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8bb626-f371-4f81-a366-a6c03020c1fc None admin] RBAC: Authorizing `identity:get_project(project_id=a7f9d22fd79e4b1fa16a2f60fc6faec7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:41.617041 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8bb626-f371-4f81-a366-a6c03020c1fc None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:41.620250 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 118/233] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:23:41 2022] GET /identity/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:41.629347 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582f1e7f-265a-4fac-9f8f-8fa8b993b7b3 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:41.780690 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582f1e7f-265a-4fac-9f8f-8fa8b993b7b3 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:41.831747 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f129c-403a-4e1f-bccf-32379cc538f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:41.832727 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f129c-403a-4e1f-bccf-32379cc538f5 None admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:41.833157 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f129c-403a-4e1f-bccf-32379cc538f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:41.833579 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f129c-403a-4e1f-bccf-32379cc538f5 None admin] PATH_INFO: `/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7/users/2cfe991c2c894beea1379ba2c7dac1a9/roles/0cdb52ffa00449399b27dbfcd383f488` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:41.838382 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9f129c-403a-4e1f-bccf-32379cc538f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=a7f9d22fd79e4b1fa16a2f60fc6faec7, user_id=2cfe991c2c894beea1379ba2c7dac1a9, role_id=0cdb52ffa00449399b27dbfcd383f488)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:41.839999 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9f129c-403a-4e1f-bccf-32379cc538f5 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:41.946689 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 116/234] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Aug 24 05:23:41 2022] PUT /identity/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7/users/2cfe991c2c894beea1379ba2c7dac1a9/roles/0cdb52ffa00449399b27dbfcd383f488 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:23:42.437623 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959136b-a0a5-477f-b88e-8e1a910acd72 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:42.439200 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959136b-a0a5-477f-b88e-8e1a910acd72 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:42.439693 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959136b-a0a5-477f-b88e-8e1a910acd72 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:42.441727 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 119/235] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:42.446892 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f2e59-15a1-4db7-9094-4ed1dcf683e4 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:42.447864 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f2e59-15a1-4db7-9094-4ed1dcf683e4 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:42.448346 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f2e59-15a1-4db7-9094-4ed1dcf683e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:42.448674 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3d969-3c2d-41a9-84ad-b3613a0ef3cf None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:42.448969 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3d969-3c2d-41a9-84ad-b3613a0ef3cf None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:42.449257 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3d969-3c2d-41a9-84ad-b3613a0ef3cf None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:42.450756 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 120/236] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:23:42.456875 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c3408-af70-4799-af74-39fdcf122266 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:42.457166 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c3408-af70-4799-af74-39fdcf122266 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:42.457454 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c3408-af70-4799-af74-39fdcf122266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:42.478493 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-0a9c3408-af70-4799-af74-39fdcf122266 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:42.479005 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-409f2e59-15a1-4db7-9094-4ed1dcf683e4 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:42.479697 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-0a9c3408-af70-4799-af74-39fdcf122266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:42.480662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-409f2e59-15a1-4db7-9094-4ed1dcf683e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:42.515764 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:42.573685 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 121/237] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:23:42.614995 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 117/238] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:42.883151 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9c3408-af70-4799-af74-39fdcf122266 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:42.915866 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:43.002897 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9c3408-af70-4799-af74-39fdcf122266 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:43.024999 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f2e59-15a1-4db7-9094-4ed1dcf683e4 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:43.045668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd745ee6-8337-4b8e-858d-514940699278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:43.046704 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd745ee6-8337-4b8e-858d-514940699278 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:43.049682 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd745ee6-8337-4b8e-858d-514940699278 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:43.050228 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd745ee6-8337-4b8e-858d-514940699278 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:43.073814 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd745ee6-8337-4b8e-858d-514940699278 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:23:43.081649 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-fd745ee6-8337-4b8e-858d-514940699278 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:23:43.081649 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:23:43.081649 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:23:43.081649 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:23:43.081649 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:23:43.081649 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:23:43.081649 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:23:43.081649 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:23:43.081649 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:23:43.081649 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:23:43.081649 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:23:43.081649 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:23:43.081649 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:23:43.081649 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:23:43.081649 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:23:43.081649 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:23:43.082527 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:23:43.082527 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:23:43.082527 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:23:43.082527 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:23:43.082527 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:23:43.082527 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:23:43.082527 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:23:43.082527 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:23:43.082527 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:23:43.082527 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:23:43.082527 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:23:43.082527 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:23:43.082527 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:23:43.082527 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:23:43.082527 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:23:43.082527 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:23:43.083973 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:23:43.083973 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:23:43.083973 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:23:43.083973 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:23:43.083973 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:23:43.083973 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:23:43.083973 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:23:43.083973 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:23:43.083973 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:23:43.083973 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:23:43.083973 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:23:43.083973 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:23:43.083973 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 122/239] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:23:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:23:43.089343 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd745ee6-8337-4b8e-858d-514940699278 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:43.166447 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f2e59-15a1-4db7-9094-4ed1dcf683e4 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:43.212640 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddbda94-1eb6-48b7-b866-257ae2754b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:43.213467 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbda94-1eb6-48b7-b866-257ae2754b37 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:43.213846 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbda94-1eb6-48b7-b866-257ae2754b37 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:43.214217 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbda94-1eb6-48b7-b866-257ae2754b37 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:43.215440 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddbda94-1eb6-48b7-b866-257ae2754b37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:43.217043 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddbda94-1eb6-48b7-b866-257ae2754b37 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:43.224148 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 118/240] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:23:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:23:43.224148 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91fd0e-e3d6-4403-98b3-dbb05358fb05 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:43.224148 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91fd0e-e3d6-4403-98b3-dbb05358fb05 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:43.224148 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91fd0e-e3d6-4403-98b3-dbb05358fb05 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:43.224148 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 119/241] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:43.229055 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91fd0e-e3d6-4403-98b3-dbb05358fb05 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:43.236283 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:43.252573 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd745ee6-8337-4b8e-858d-514940699278 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:43.255828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:43.290411 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b2d017-c598-4943-ad8d-c6060aea7e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:43.291318 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2d017-c598-4943-ad8d-c6060aea7e4a None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:43.291777 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2d017-c598-4943-ad8d-c6060aea7e4a None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:43.292225 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2d017-c598-4943-ad8d-c6060aea7e4a None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:43.293117 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b2d017-c598-4943-ad8d-c6060aea7e4a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:43.293598 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b2d017-c598-4943-ad8d-c6060aea7e4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:43.295249 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b2d017-c598-4943-ad8d-c6060aea7e4a None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:43.305332 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 123/242] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Aug 24 05:23:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:43.306938 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955becc-9b90-4c48-b98b-1d306991e787 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:43.307426 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955becc-9b90-4c48-b98b-1d306991e787 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:43.307933 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955becc-9b90-4c48-b98b-1d306991e787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:43.328244 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-9955becc-9b90-4c48-b98b-1d306991e787 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:43.330458 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-9955becc-9b90-4c48-b98b-1d306991e787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:43.373650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91fd0e-e3d6-4403-98b3-dbb05358fb05 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:43.414301 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc1701-f2e9-4288-9296-c8922cd9bca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:43.415123 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc1701-f2e9-4288-9296-c8922cd9bca4 None admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:43.415230 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc1701-f2e9-4288-9296-c8922cd9bca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:43.415456 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc1701-f2e9-4288-9296-c8922cd9bca4 None admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:43.416115 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dc1701-f2e9-4288-9296-c8922cd9bca4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:43.417746 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dc1701-f2e9-4288-9296-c8922cd9bca4 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:43.429330 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 124/243] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:43.429536 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9955becc-9b90-4c48-b98b-1d306991e787 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:43.528005 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 120/244] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Aug 24 05:23:43 2022] POST /identity/v3/users => generated 293 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:23:43.548246 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:43.582467 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9955becc-9b90-4c48-b98b-1d306991e787 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:43.631774 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a05a4b-8e75-4416-963a-08b002a3ff0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:43.632442 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a05a4b-8e75-4416-963a-08b002a3ff0a None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:43.632778 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a05a4b-8e75-4416-963a-08b002a3ff0a None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:43.633062 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a05a4b-8e75-4416-963a-08b002a3ff0a None admin] PATH_INFO: `/v3/users/6ef5ef2364e842b1bf49b6df0f65d06e` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:43.657657 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a05a4b-8e75-4416-963a-08b002a3ff0a None admin] RBAC: Authorizing `identity:get_user(user_id=6ef5ef2364e842b1bf49b6df0f65d06e)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:43.659506 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a05a4b-8e75-4416-963a-08b002a3ff0a None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:43.661326 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 125/245] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:23:43 2022] GET /identity/v3/users/6ef5ef2364e842b1bf49b6df0f65d06e => generated 297 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:43.670628 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc1701-f2e9-4288-9296-c8922cd9bca4 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:43.687686 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a81c7-2c9e-4f4c-9841-bd43a5e93f34 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:43.688100 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a81c7-2c9e-4f4c-9841-bd43a5e93f34 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:43.688569 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a81c7-2c9e-4f4c-9841-bd43a5e93f34 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:43.690451 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 126/246] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:23:43.695897 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec78fc-d988-4bc7-81c3-bd6943b753c3 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:43.696183 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec78fc-d988-4bc7-81c3-bd6943b753c3 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:43.696487 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec78fc-d988-4bc7-81c3-bd6943b753c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:43.722997 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-6dec78fc-d988-4bc7-81c3-bd6943b753c3 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:43.724238 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-6dec78fc-d988-4bc7-81c3-bd6943b753c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:43.795899 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:43.829663 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc1701-f2e9-4288-9296-c8922cd9bca4 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:43.842527 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 127/247] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:23:43.843389 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dec78fc-d988-4bc7-81c3-bd6943b753c3 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:43.867187 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd105d-511f-4153-ad74-030d5ec4bc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:43.868268 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd105d-511f-4153-ad74-030d5ec4bc9c None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:43.868752 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd105d-511f-4153-ad74-030d5ec4bc9c None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:43.869210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd105d-511f-4153-ad74-030d5ec4bc9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:43.912080 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:43.958270 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dec78fc-d988-4bc7-81c3-bd6943b753c3 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:43.959419 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cd105d-511f-4153-ad74-030d5ec4bc9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:43.961048 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cd105d-511f-4153-ad74-030d5ec4bc9c None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:44.007728 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc646285-04f6-4de8-8e0e-7478333f7fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:44.008409 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc646285-04f6-4de8-8e0e-7478333f7fe2 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:44.009233 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc646285-04f6-4de8-8e0e-7478333f7fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:44.009233 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc646285-04f6-4de8-8e0e-7478333f7fe2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:44.022704 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc646285-04f6-4de8-8e0e-7478333f7fe2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:23:44.028095 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-bc646285-04f6-4de8-8e0e-7478333f7fe2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:23:44.028095 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:23:44.028095 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:23:44.028095 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:23:44.028095 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:23:44.028095 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:23:44.028095 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:23:44.028095 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:23:44.028095 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:23:44.028095 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:23:44.028095 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:23:44.028095 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:23:44.028095 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:23:44.028095 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:23:44.028095 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:23:44.028095 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:23:44.029042 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:23:44.029042 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:23:44.029042 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:23:44.029042 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:23:44.029042 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:23:44.029042 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:23:44.029042 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:23:44.029042 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:23:44.029042 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:23:44.029042 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:23:44.029042 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:23:44.029042 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:23:44.029042 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:23:44.029042 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:23:44.029042 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:23:44.029042 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:23:44.029820 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:23:44.029820 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:23:44.029820 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:23:44.029820 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:23:44.029820 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:23:44.029820 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:23:44.029820 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:23:44.029820 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:23:44.029820 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:23:44.029820 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:23:44.029820 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:23:44.029820 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:23:44.030422 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 128/248] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:23:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:23:44.040306 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc646285-04f6-4de8-8e0e-7478333f7fe2 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:44.076384 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:44.111922 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 121/249] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:23:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:23:44.118747 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd105d-511f-4153-ad74-030d5ec4bc9c None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:44.196905 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc646285-04f6-4de8-8e0e-7478333f7fe2 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:44.232102 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:44.243164 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d75f928-beff-458a-940b-a8cfc1ab5299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:44.243786 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75f928-beff-458a-940b-a8cfc1ab5299 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:44.244064 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75f928-beff-458a-940b-a8cfc1ab5299 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:44.245334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75f928-beff-458a-940b-a8cfc1ab5299 None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:44.245334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d75f928-beff-458a-940b-a8cfc1ab5299 None admin] RBAC: Adding query filter params (name=member) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:44.245809 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d75f928-beff-458a-940b-a8cfc1ab5299 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:44.247390 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d75f928-beff-458a-940b-a8cfc1ab5299 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:44.258235 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 129/250] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Aug 24 05:23:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:44.259679 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d75f928-beff-458a-940b-a8cfc1ab5299 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:44.264922 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd105d-511f-4153-ad74-030d5ec4bc9c None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:44.322616 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb81b1-c587-4087-92f6-f3efba222a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:44.323235 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb81b1-c587-4087-92f6-f3efba222a7e None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:44.323524 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb81b1-c587-4087-92f6-f3efba222a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:44.323809 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb81b1-c587-4087-92f6-f3efba222a7e None admin] PATH_INFO: `/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:44.324989 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fb81b1-c587-4087-92f6-f3efba222a7e None admin] RBAC: Authorizing `identity:get_project(project_id=a7f9d22fd79e4b1fa16a2f60fc6faec7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:44.326664 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fb81b1-c587-4087-92f6-f3efba222a7e None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:44.328418 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 122/251] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:23:44 2022] GET /identity/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:44.329200 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb81b1-c587-4087-92f6-f3efba222a7e None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:44.380528 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:44.441894 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb81b1-c587-4087-92f6-f3efba222a7e None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:44.477513 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc280c-162d-4c06-ac67-467776efcdfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:44.478166 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc280c-162d-4c06-ac67-467776efcdfc None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:44.478452 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc280c-162d-4c06-ac67-467776efcdfc None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:44.478734 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc280c-162d-4c06-ac67-467776efcdfc None admin] PATH_INFO: `/v3/users/2117f4e51a7d4a7f9295f2f8c12e8713` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:44.479896 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc280c-162d-4c06-ac67-467776efcdfc None admin] RBAC: Authorizing `identity:get_user(user_id=2117f4e51a7d4a7f9295f2f8c12e8713)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:44.481432 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc280c-162d-4c06-ac67-467776efcdfc None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:44.483106 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 123/252] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:23:44 2022] GET /identity/v3/users/2117f4e51a7d4a7f9295f2f8c12e8713 => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:44.483889 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc280c-162d-4c06-ac67-467776efcdfc None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:44.491824 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d75f928-beff-458a-940b-a8cfc1ab5299 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:44.565163 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b9132-5646-46b9-8bd9-7192dd792fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:44.565810 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b9132-5646-46b9-8bd9-7192dd792fa2 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:44.566094 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b9132-5646-46b9-8bd9-7192dd792fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:44.566376 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b9132-5646-46b9-8bd9-7192dd792fa2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:44.590557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc280c-162d-4c06-ac67-467776efcdfc None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:44.597951 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22b9132-5646-46b9-8bd9-7192dd792fa2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:23:44.603279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-a22b9132-5646-46b9-8bd9-7192dd792fa2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:23:44.603279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:23:44.603279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:23:44.603279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:23:44.603279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:23:44.603279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:23:44.603279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:23:44.603279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:23:44.603279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:23:44.603279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:23:44.603279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:23:44.603279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:23:44.603279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:23:44.603279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:23:44.603279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:23:44.603279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:23:44.604242 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:23:44.604242 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:23:44.604242 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:23:44.604242 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:23:44.604242 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:23:44.604242 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:23:44.604242 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:23:44.604242 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:23:44.604242 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:23:44.604242 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:23:44.604242 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:23:44.604242 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:23:44.604242 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:23:44.604242 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:23:44.604242 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:23:44.604242 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:23:44.605064 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:23:44.605064 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:23:44.605064 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:23:44.605064 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:23:44.605064 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:23:44.605064 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:23:44.605064 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:23:44.605064 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:23:44.605064 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:23:44.605064 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:23:44.605064 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:23:44.605064 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:23:44.606526 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:44.607153 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 130/253] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Aug 24 05:23:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:23:44.608026 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b9132-5646-46b9-8bd9-7192dd792fa2 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:44.631400 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec55fbc-cb0b-4348-88a4-6e03b4bf598a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:44.632024 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec55fbc-cb0b-4348-88a4-6e03b4bf598a None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:44.632330 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec55fbc-cb0b-4348-88a4-6e03b4bf598a None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:44.633189 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec55fbc-cb0b-4348-88a4-6e03b4bf598a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:44.633939 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec55fbc-cb0b-4348-88a4-6e03b4bf598a None admin] RBAC: Adding query filter params (role.id=49b52d60e4434f05870a37c49b2e6e6b, scope.project.id=a7f9d22fd79e4b1fa16a2f60fc6faec7, user.id=6ef5ef2364e842b1bf49b6df0f65d06e) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:44.634253 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec55fbc-cb0b-4348-88a4-6e03b4bf598a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:44.635703 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec55fbc-cb0b-4348-88a4-6e03b4bf598a None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:44.643119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 124/254] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:23:44 2022] GET /identity/v3/role_assignments?user.id=6ef5ef2364e842b1bf49b6df0f65d06e&scope.project.id=a7f9d22fd79e4b1fa16a2f60fc6faec7&role.id=49b52d60e4434f05870a37c49b2e6e6b => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:44.645337 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac89afb-f316-4c69-96f2-1fcfc7cd3f4d None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:44.646143 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac89afb-f316-4c69-96f2-1fcfc7cd3f4d None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:44.646589 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac89afb-f316-4c69-96f2-1fcfc7cd3f4d None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:44.648060 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 125/255] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:44.651238 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac89afb-f316-4c69-96f2-1fcfc7cd3f4d None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:44.685184 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:44.796392 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b9132-5646-46b9-8bd9-7192dd792fa2 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:44.811104 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac89afb-f316-4c69-96f2-1fcfc7cd3f4d None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:44.856317 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35268d8-d516-4fbe-a699-44e6efa89552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:44.857046 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35268d8-d516-4fbe-a699-44e6efa89552 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:44.857327 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35268d8-d516-4fbe-a699-44e6efa89552 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:44.857602 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35268d8-d516-4fbe-a699-44e6efa89552 None admin] PATH_INFO: `/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:44.859723 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35268d8-d516-4fbe-a699-44e6efa89552 None admin] RBAC: Authorizing `identity:get_project(project_id=a7f9d22fd79e4b1fa16a2f60fc6faec7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:44.862021 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35268d8-d516-4fbe-a699-44e6efa89552 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:44.863849 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 131/256] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:23:44 2022] GET /identity/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7 => generated 302 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:44.866023 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd8c245-4a4d-48f9-b23d-0b51f03d36e0 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:44.866307 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd8c245-4a4d-48f9-b23d-0b51f03d36e0 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:44.866576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd8c245-4a4d-48f9-b23d-0b51f03d36e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:44.890319 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e1372-e4c3-4d10-8824-e858ed4f9b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:44.891580 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e1372-e4c3-4d10-8824-e858ed4f9b8e None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:44.891919 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e1372-e4c3-4d10-8824-e858ed4f9b8e None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:44.892207 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e1372-e4c3-4d10-8824-e858ed4f9b8e None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:44.892960 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e1372-e4c3-4d10-8824-e858ed4f9b8e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:44.893267 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e1372-e4c3-4d10-8824-e858ed4f9b8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:44.895709 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e1372-e4c3-4d10-8824-e858ed4f9b8e None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:44.901670 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-7fd8c245-4a4d-48f9-b23d-0b51f03d36e0 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:44.902863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-7fd8c245-4a4d-48f9-b23d-0b51f03d36e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:44.912670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 126/257] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:23:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:44.913511 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e1372-e4c3-4d10-8824-e858ed4f9b8e None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:44.961686 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:45.019678 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 132/258] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:45.020588 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd8c245-4a4d-48f9-b23d-0b51f03d36e0 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:45.076870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e1372-e4c3-4d10-8824-e858ed4f9b8e None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:45.120843 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:45.134000 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31352f97-9c01-4cb4-badf-8fca228d3b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:45.134682 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31352f97-9c01-4cb4-badf-8fca228d3b67 None admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:45.134963 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31352f97-9c01-4cb4-badf-8fca228d3b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:45.137698 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31352f97-9c01-4cb4-badf-8fca228d3b67 None admin] PATH_INFO: `/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7/users/2117f4e51a7d4a7f9295f2f8c12e8713/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:45.137698 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31352f97-9c01-4cb4-badf-8fca228d3b67 None admin] RBAC: Authorizing `identity:create_grant(project_id=a7f9d22fd79e4b1fa16a2f60fc6faec7, user_id=2117f4e51a7d4a7f9295f2f8c12e8713, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:45.138724 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31352f97-9c01-4cb4-badf-8fca228d3b67 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:45.196742 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd8c245-4a4d-48f9-b23d-0b51f03d36e0 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:45.251590 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9809804c-8c2f-4ef4-b07b-cd7ec39d044d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:45.252230 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9809804c-8c2f-4ef4-b07b-cd7ec39d044d None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:45.252535 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9809804c-8c2f-4ef4-b07b-cd7ec39d044d None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:45.252802 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9809804c-8c2f-4ef4-b07b-cd7ec39d044d None admin] PATH_INFO: `/v3/users/2cfe991c2c894beea1379ba2c7dac1a9` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:45.257076 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9809804c-8c2f-4ef4-b07b-cd7ec39d044d None admin] RBAC: Authorizing `identity:get_user(user_id=2cfe991c2c894beea1379ba2c7dac1a9)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:45.258537 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9809804c-8c2f-4ef4-b07b-cd7ec39d044d None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:45.260187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 133/259] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:23:45 2022] GET /identity/v3/users/2cfe991c2c894beea1379ba2c7dac1a9 => generated 283 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:45.265776 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9809804c-8c2f-4ef4-b07b-cd7ec39d044d None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:45.287832 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 127/260] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Aug 24 05:23:44 2022] PUT /identity/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7/users/2117f4e51a7d4a7f9295f2f8c12e8713/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:23:45.378939 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31352f97-9c01-4cb4-badf-8fca228d3b67 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:45.395031 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9809804c-8c2f-4ef4-b07b-cd7ec39d044d None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:45.412800 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:45.435598 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e874a36-1ffb-430a-89d2-829033b8627c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:45.436224 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e874a36-1ffb-430a-89d2-829033b8627c None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:45.436536 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e874a36-1ffb-430a-89d2-829033b8627c None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:45.436815 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e874a36-1ffb-430a-89d2-829033b8627c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:45.518644 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e874a36-1ffb-430a-89d2-829033b8627c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:45.520332 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e874a36-1ffb-430a-89d2-829033b8627c None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:45.612320 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:45.626922 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31352f97-9c01-4cb4-badf-8fca228d3b67 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:45.633937 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 134/261] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:23:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:23:45.651728 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e874a36-1ffb-430a-89d2-829033b8627c None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:45.668653 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a11b95e-7f9d-4ecd-8192-d324744789e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:45.669330 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11b95e-7f9d-4ecd-8192-d324744789e4 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:45.669607 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11b95e-7f9d-4ecd-8192-d324744789e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:45.669930 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11b95e-7f9d-4ecd-8192-d324744789e4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:45.678352 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:45.683445 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a11b95e-7f9d-4ecd-8192-d324744789e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:23:45.689295 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-4a11b95e-7f9d-4ecd-8192-d324744789e4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:23:45.689295 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:23:45.689295 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:23:45.689295 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:23:45.689295 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:23:45.689295 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:23:45.689295 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:23:45.689295 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:23:45.689295 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:23:45.689295 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:23:45.689295 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:23:45.689295 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:23:45.689295 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:23:45.689295 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:23:45.689295 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:23:45.689295 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:23:45.695445 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:23:45.695445 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:23:45.695445 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:23:45.695445 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:23:45.695445 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:23:45.695445 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:23:45.695445 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:23:45.695445 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:23:45.695445 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:23:45.695445 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:23:45.695445 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:23:45.695445 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:23:45.695445 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:23:45.695445 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:23:45.695445 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:23:45.695445 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:23:45.696232 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:23:45.696232 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:23:45.696232 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:23:45.696232 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:23:45.696232 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:23:45.696232 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:23:45.696232 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:23:45.696232 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:23:45.696232 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:23:45.696232 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:23:45.696232 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:23:45.696232 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:23:45.696232 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 128/262] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Aug 24 05:23:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:23:45.700420 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a11b95e-7f9d-4ecd-8192-d324744789e4 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:45.804321 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a11b95e-7f9d-4ecd-8192-d324744789e4 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:45.840438 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a6701-aec7-4ba2-941b-b0275f13f3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:45.841246 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a6701-aec7-4ba2-941b-b0275f13f3cc None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:45.841596 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a6701-aec7-4ba2-941b-b0275f13f3cc None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:45.841940 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a6701-aec7-4ba2-941b-b0275f13f3cc None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:45.842708 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64a6701-aec7-4ba2-941b-b0275f13f3cc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:45.843084 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64a6701-aec7-4ba2-941b-b0275f13f3cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:45.845160 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64a6701-aec7-4ba2-941b-b0275f13f3cc None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:45.858265 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 129/263] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:23:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:23:45.859435 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a6701-aec7-4ba2-941b-b0275f13f3cc None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:45.879424 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:45.909508 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e874a36-1ffb-430a-89d2-829033b8627c None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:45.966235 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a6701-aec7-4ba2-941b-b0275f13f3cc None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:45.974762 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf36ebf-f570-4ccf-85ab-5b57dee347d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:45.975741 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf36ebf-f570-4ccf-85ab-5b57dee347d1 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:45.976247 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf36ebf-f570-4ccf-85ab-5b57dee347d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:45.976803 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf36ebf-f570-4ccf-85ab-5b57dee347d1 None admin] PATH_INFO: `/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:45.981327 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf36ebf-f570-4ccf-85ab-5b57dee347d1 None admin] RBAC: Authorizing `identity:get_project(project_id=a7f9d22fd79e4b1fa16a2f60fc6faec7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:45.983025 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf36ebf-f570-4ccf-85ab-5b57dee347d1 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:45.985527 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 135/264] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:23:45 2022] GET /identity/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7 => generated 302 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:23:46.006088 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf36ebf-f570-4ccf-85ab-5b57dee347d1 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:46.008343 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c44370-d014-4344-b12f-16add426ae8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:46.009021 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c44370-d014-4344-b12f-16add426ae8e None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:46.009299 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c44370-d014-4344-b12f-16add426ae8e None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:46.009574 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c44370-d014-4344-b12f-16add426ae8e None admin] PATH_INFO: `/v3/users/2cfe991c2c894beea1379ba2c7dac1a9` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:46.016425 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c44370-d014-4344-b12f-16add426ae8e None admin] RBAC: Authorizing `identity:get_user(user_id=2cfe991c2c894beea1379ba2c7dac1a9)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:46.016425 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c44370-d014-4344-b12f-16add426ae8e None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:46.016425 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 130/265] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:23:45 2022] GET /identity/v3/users/2cfe991c2c894beea1379ba2c7dac1a9 => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:23:46.020892 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c44370-d014-4344-b12f-16add426ae8e None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:46.121051 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:46.138177 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c44370-d014-4344-b12f-16add426ae8e None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:46.183435 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f17283-ed7e-440f-8b4a-af7989199f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:46.183435 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f17283-ed7e-440f-8b4a-af7989199f14 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:46.183435 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f17283-ed7e-440f-8b4a-af7989199f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:46.183435 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f17283-ed7e-440f-8b4a-af7989199f14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:46.193341 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:46.217617 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf36ebf-f570-4ccf-85ab-5b57dee347d1 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:46.268981 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f17283-ed7e-440f-8b4a-af7989199f14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:46.270488 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f17283-ed7e-440f-8b4a-af7989199f14 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:46.289177 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd9dd18-3127-4b07-9053-800af145036c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:46.290201 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9dd18-3127-4b07-9053-800af145036c None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:46.290810 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9dd18-3127-4b07-9053-800af145036c None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:46.291099 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9dd18-3127-4b07-9053-800af145036c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:46.291853 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd9dd18-3127-4b07-9053-800af145036c None admin] RBAC: Adding query filter params (role.id=e013ad4d436f4cb0a9bf7debb7d13dc7, scope.project.id=a7f9d22fd79e4b1fa16a2f60fc6faec7, user.id=2cfe991c2c894beea1379ba2c7dac1a9) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:46.292227 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd9dd18-3127-4b07-9053-800af145036c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:46.293739 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd9dd18-3127-4b07-9053-800af145036c None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:46.304889 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 136/266] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:23:46 2022] GET /identity/v3/role_assignments?user.id=2cfe991c2c894beea1379ba2c7dac1a9&scope.project.id=a7f9d22fd79e4b1fa16a2f60fc6faec7&role.id=e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 608 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:46.404814 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:46.426248 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 131/267] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:23:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 05:23:46.433627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd9dd18-3127-4b07-9053-800af145036c None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:46.580748 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd9dd18-3127-4b07-9053-800af145036c None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:46.618767 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f922f1c3-ab3a-4f87-ad51-b614949fa098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:46.619424 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f922f1c3-ab3a-4f87-ad51-b614949fa098 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:46.619847 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f922f1c3-ab3a-4f87-ad51-b614949fa098 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:46.620127 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f922f1c3-ab3a-4f87-ad51-b614949fa098 None admin] PATH_INFO: `/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:46.621310 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f922f1c3-ab3a-4f87-ad51-b614949fa098 None admin] RBAC: Authorizing `identity:get_project(project_id=a7f9d22fd79e4b1fa16a2f60fc6faec7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:46.622879 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f922f1c3-ab3a-4f87-ad51-b614949fa098 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:46.624606 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 137/268] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:23:46 2022] GET /identity/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:46.631666 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f17283-ed7e-440f-8b4a-af7989199f14 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:46.696044 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f508009-88ee-4548-8c38-5438d54e7803 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:46.696546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f508009-88ee-4548-8c38-5438d54e7803 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:46.696546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f508009-88ee-4548-8c38-5438d54e7803 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:46.698147 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 138/269] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:46.709892 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee32c2-1c91-480b-946a-b9c85263c1ac None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:46.710183 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee32c2-1c91-480b-946a-b9c85263c1ac None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:46.710439 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee32c2-1c91-480b-946a-b9c85263c1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:46.744856 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-9fee32c2-1c91-480b-946a-b9c85263c1ac None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:46.746143 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-9fee32c2-1c91-480b-946a-b9c85263c1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:46.760108 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f17283-ed7e-440f-8b4a-af7989199f14 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:46.769068 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:46.804443 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6404fa3-ff14-4580-9b23-86d75e1ea188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:46.805889 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6404fa3-ff14-4580-9b23-86d75e1ea188 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:46.806173 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6404fa3-ff14-4580-9b23-86d75e1ea188 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:46.806452 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6404fa3-ff14-4580-9b23-86d75e1ea188 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:46.807189 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6404fa3-ff14-4580-9b23-86d75e1ea188 None admin] RBAC: Adding query filter params (role.id=0cdb52ffa00449399b27dbfcd383f488, scope.project.id=a7f9d22fd79e4b1fa16a2f60fc6faec7, user.id=2cfe991c2c894beea1379ba2c7dac1a9) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:46.807504 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6404fa3-ff14-4580-9b23-86d75e1ea188 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:46.809009 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6404fa3-ff14-4580-9b23-86d75e1ea188 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:46.820507 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 132/270] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:23:46 2022] GET /identity/v3/role_assignments?user.id=2cfe991c2c894beea1379ba2c7dac1a9&scope.project.id=a7f9d22fd79e4b1fa16a2f60fc6faec7&role.id=0cdb52ffa00449399b27dbfcd383f488 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:46.879016 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 139/271] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:46.988552 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5abe7f0-7b4f-412a-aad7-5f54e7a2dcac None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:46.989152 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5abe7f0-7b4f-412a-aad7-5f54e7a2dcac None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:46.989666 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5abe7f0-7b4f-412a-aad7-5f54e7a2dcac None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:46.991456 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 133/272] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:47.002682 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dfc780-835b-4282-9192-038863bdd95a None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:47.002993 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dfc780-835b-4282-9192-038863bdd95a None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:47.003231 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dfc780-835b-4282-9192-038863bdd95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:47.028351 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-56dfc780-835b-4282-9192-038863bdd95a None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:47.029572 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-56dfc780-835b-4282-9192-038863bdd95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:47.108852 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 140/273] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:47.238391 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5abe7f0-7b4f-412a-aad7-5f54e7a2dcac None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:47.300268 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:47.354638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5abe7f0-7b4f-412a-aad7-5f54e7a2dcac None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:47.389728 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea925303-1d3d-4f5d-a167-810c15595966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:47.390363 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea925303-1d3d-4f5d-a167-810c15595966 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:47.390641 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea925303-1d3d-4f5d-a167-810c15595966 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:47.390943 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea925303-1d3d-4f5d-a167-810c15595966 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:47.392130 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea925303-1d3d-4f5d-a167-810c15595966 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:47.393612 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea925303-1d3d-4f5d-a167-810c15595966 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:47.395518 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 134/274] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:23:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:47.400800 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dfc780-835b-4282-9192-038863bdd95a None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:47.423683 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea925303-1d3d-4f5d-a167-810c15595966 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:47.490215 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:47.516510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dfc780-835b-4282-9192-038863bdd95a None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:47.531049 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea925303-1d3d-4f5d-a167-810c15595966 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:47.550655 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:47.557643 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3797cfcb-5424-4356-9e98-a3ac101bc7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:47.558370 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797cfcb-5424-4356-9e98-a3ac101bc7e1 None admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:47.558733 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797cfcb-5424-4356-9e98-a3ac101bc7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:47.559013 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797cfcb-5424-4356-9e98-a3ac101bc7e1 None admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:47.561528 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3797cfcb-5424-4356-9e98-a3ac101bc7e1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:47.561528 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3797cfcb-5424-4356-9e98-a3ac101bc7e1 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:47.567683 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cdf561-7f53-4a52-93e5-d6a9262ef0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:47.568315 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cdf561-7f53-4a52-93e5-d6a9262ef0e4 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:47.568622 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cdf561-7f53-4a52-93e5-d6a9262ef0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:47.568897 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cdf561-7f53-4a52-93e5-d6a9262ef0e4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:47.580371 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cdf561-7f53-4a52-93e5-d6a9262ef0e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:23:47.585319 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-37cdf561-7f53-4a52-93e5-d6a9262ef0e4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:23:47.585319 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:23:47.585319 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:23:47.585319 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:23:47.585319 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:23:47.585319 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:23:47.585319 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:23:47.585319 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:23:47.585319 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:23:47.585319 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:23:47.585319 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:23:47.585319 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:23:47.585319 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:23:47.585319 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:23:47.585319 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:23:47.585319 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:23:47.586381 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:23:47.586381 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:23:47.586381 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:23:47.586381 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:23:47.586381 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:23:47.586381 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:23:47.586381 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:23:47.586381 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:23:47.586381 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:23:47.586381 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:23:47.586381 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:23:47.586381 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:23:47.586381 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:23:47.586381 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:23:47.586381 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:23:47.586381 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:23:47.587492 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:23:47.587492 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:23:47.587492 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:23:47.587492 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:23:47.587492 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:23:47.587492 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:23:47.587492 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:23:47.587492 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:23:47.587492 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:23:47.587492 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:23:47.587492 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:23:47.587492 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:23:47.587492 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 135/275] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:23:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:23:47.588195 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc290b-6584-4f24-b9b9-c53373293254 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:47.588195 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc290b-6584-4f24-b9b9-c53373293254 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:47.588469 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc290b-6584-4f24-b9b9-c53373293254 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:47.590135 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 136/276] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:47.595580 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8886ee-c34e-4677-9661-011175e51084 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:47.601919 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8886ee-c34e-4677-9661-011175e51084 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:47.601919 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8886ee-c34e-4677-9661-011175e51084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:47.615714 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-2f8886ee-c34e-4677-9661-011175e51084 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:47.616995 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-2f8886ee-c34e-4677-9661-011175e51084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:47.687316 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 137/277] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:23:47.697028 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 141/278] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Aug 24 05:23:47 2022] POST /identity/v3/users => generated 293 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:23:47.697028 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3797cfcb-5424-4356-9e98-a3ac101bc7e1 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:47.827467 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3797cfcb-5424-4356-9e98-a3ac101bc7e1 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:47.861803 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:47.869516 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a72a5-0517-4395-bb4b-6e6519b1e86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:47.870146 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a72a5-0517-4395-bb4b-6e6519b1e86a None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:47.876269 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a72a5-0517-4395-bb4b-6e6519b1e86a None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:47.876269 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a72a5-0517-4395-bb4b-6e6519b1e86a None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:47.876269 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a72a5-0517-4395-bb4b-6e6519b1e86a None admin] RBAC: Adding query filter params (name=member) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:47.876269 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a72a5-0517-4395-bb4b-6e6519b1e86a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:47.876269 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a72a5-0517-4395-bb4b-6e6519b1e86a None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:47.883342 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 142/279] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Aug 24 05:23:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:47.888780 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8886ee-c34e-4677-9661-011175e51084 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:47.973479 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a72a5-0517-4395-bb4b-6e6519b1e86a None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:48.010876 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8886ee-c34e-4677-9661-011175e51084 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:48.024288 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:48.061366 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90277bd-d468-4dd9-96f8-9766882eb43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:48.062014 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90277bd-d468-4dd9-96f8-9766882eb43f None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:48.062457 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90277bd-d468-4dd9-96f8-9766882eb43f None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:48.062780 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90277bd-d468-4dd9-96f8-9766882eb43f None admin] PATH_INFO: `/v3/users/0f2c9f2165a04d738e90d3a7f7deb767` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:48.082801 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90277bd-d468-4dd9-96f8-9766882eb43f None admin] RBAC: Authorizing `identity:get_user(user_id=0f2c9f2165a04d738e90d3a7f7deb767)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:48.085079 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90277bd-d468-4dd9-96f8-9766882eb43f None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:48.090127 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 138/280] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:23:47 2022] GET /identity/v3/users/0f2c9f2165a04d738e90d3a7f7deb767 => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:48.090127 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613893ac-c785-4b03-9d31-915ae51e63fd None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:48.090127 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613893ac-c785-4b03-9d31-915ae51e63fd None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:48.090127 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613893ac-c785-4b03-9d31-915ae51e63fd None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:48.091320 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 139/281] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:48.097448 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f340c4-ab3a-4743-ab35-daa561b2b85e None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:48.097740 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f340c4-ab3a-4743-ab35-daa561b2b85e None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:48.098024 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f340c4-ab3a-4743-ab35-daa561b2b85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:48.110042 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a72a5-0517-4395-bb4b-6e6519b1e86a None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:48.117435 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-b9f340c4-ab3a-4743-ab35-daa561b2b85e None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:48.118776 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-b9f340c4-ab3a-4743-ab35-daa561b2b85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:48.151145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728c85c7-ccd7-42ef-8def-54549306b22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:48.152392 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728c85c7-ccd7-42ef-8def-54549306b22e None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:48.153153 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728c85c7-ccd7-42ef-8def-54549306b22e None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:48.154064 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728c85c7-ccd7-42ef-8def-54549306b22e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:48.169607 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728c85c7-ccd7-42ef-8def-54549306b22e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:23:48.185736 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-728c85c7-ccd7-42ef-8def-54549306b22e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:23:48.185736 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:23:48.185736 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:23:48.185736 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:23:48.185736 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:23:48.185736 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:23:48.185736 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:23:48.185736 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:23:48.185736 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:23:48.185736 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:23:48.185736 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:23:48.185736 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:23:48.185736 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:23:48.185736 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:23:48.185736 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:23:48.185736 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:23:48.186668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:23:48.186668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:23:48.186668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:23:48.186668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:23:48.186668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:23:48.186668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:23:48.186668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:23:48.186668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:23:48.186668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:23:48.186668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:23:48.186668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:23:48.186668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:23:48.186668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:23:48.186668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:23:48.186668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:23:48.186668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:23:48.188994 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:23:48.188994 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:23:48.188994 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:23:48.188994 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:23:48.188994 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:23:48.188994 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:23:48.188994 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:23:48.188994 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:23:48.188994 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:23:48.188994 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:23:48.188994 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:23:48.188994 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:23:48.191902 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 143/282] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:23:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:23:48.193556 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728c85c7-ccd7-42ef-8def-54549306b22e None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:48.204551 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 140/283] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:48.205651 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f340c4-ab3a-4743-ab35-daa561b2b85e None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:48.284805 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:48.325542 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728c85c7-ccd7-42ef-8def-54549306b22e None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:48.373097 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c10b3e5-4cdb-48d7-a140-7b36b7b1407c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:48.373716 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10b3e5-4cdb-48d7-a140-7b36b7b1407c None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:48.373994 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10b3e5-4cdb-48d7-a140-7b36b7b1407c None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:48.374267 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10b3e5-4cdb-48d7-a140-7b36b7b1407c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:48.374996 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c10b3e5-4cdb-48d7-a140-7b36b7b1407c None admin] RBAC: Adding query filter params (role.id=87862cd6eaeb4d5fae089a801aac4f87, user.id=0f2c9f2165a04d738e90d3a7f7deb767, scope.system=all) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:48.375337 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c10b3e5-4cdb-48d7-a140-7b36b7b1407c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:48.379101 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c10b3e5-4cdb-48d7-a140-7b36b7b1407c None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:48.389185 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 144/284] 10.222.0.15 () {56 vars in 1308 bytes} [Wed Aug 24 05:23:48 2022] GET /identity/v3/role_assignments?user.id=0f2c9f2165a04d738e90d3a7f7deb767&scope.system=all&role.id=87862cd6eaeb4d5fae089a801aac4f87 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:48.397580 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:48.403870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f340c4-ab3a-4743-ab35-daa561b2b85e None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:48.443038 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f95a86-07cc-4794-a7c3-4b6972601b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:48.443657 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f95a86-07cc-4794-a7c3-4b6972601b24 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:48.443936 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f95a86-07cc-4794-a7c3-4b6972601b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:48.444211 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f95a86-07cc-4794-a7c3-4b6972601b24 None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:48.444940 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f95a86-07cc-4794-a7c3-4b6972601b24 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:48.445242 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f95a86-07cc-4794-a7c3-4b6972601b24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:48.446673 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f95a86-07cc-4794-a7c3-4b6972601b24 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:48.457154 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 141/285] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Aug 24 05:23:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:48.465586 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c10b3e5-4cdb-48d7-a140-7b36b7b1407c None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:48.571634 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f95a86-07cc-4794-a7c3-4b6972601b24 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:48.580078 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c10b3e5-4cdb-48d7-a140-7b36b7b1407c None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:48.608847 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:48.633709 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db6d0b9-dccf-44c7-9379-e1bd639aeb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:48.633709 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6d0b9-dccf-44c7-9379-e1bd639aeb20 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:48.633709 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6d0b9-dccf-44c7-9379-e1bd639aeb20 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:48.633709 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6d0b9-dccf-44c7-9379-e1bd639aeb20 None admin] PATH_INFO: `/v3/users/6ef5ef2364e842b1bf49b6df0f65d06e` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:48.634408 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db6d0b9-dccf-44c7-9379-e1bd639aeb20 None admin] RBAC: Authorizing `identity:get_user(user_id=6ef5ef2364e842b1bf49b6df0f65d06e)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:48.634408 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db6d0b9-dccf-44c7-9379-e1bd639aeb20 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:48.634408 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 145/286] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:23:48 2022] GET /identity/v3/users/6ef5ef2364e842b1bf49b6df0f65d06e => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:48.638694 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db6d0b9-dccf-44c7-9379-e1bd639aeb20 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:48.728544 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f95a86-07cc-4794-a7c3-4b6972601b24 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:48.754629 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db6d0b9-dccf-44c7-9379-e1bd639aeb20 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:48.794110 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dbe639-c3ea-4710-b6c8-1eb4d34bf970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:48.794742 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbe639-c3ea-4710-b6c8-1eb4d34bf970 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:48.795021 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbe639-c3ea-4710-b6c8-1eb4d34bf970 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:48.795297 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbe639-c3ea-4710-b6c8-1eb4d34bf970 None admin] PATH_INFO: `/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:48.799557 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dbe639-c3ea-4710-b6c8-1eb4d34bf970 None admin] RBAC: Authorizing `identity:get_project(project_id=a7f9d22fd79e4b1fa16a2f60fc6faec7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:48.801034 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dbe639-c3ea-4710-b6c8-1eb4d34bf970 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:48.802921 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 146/287] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:23:48 2022] GET /identity/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:23:48.807429 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f9e05-b5e7-46e2-9011-d1c2fb26fd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:48.808068 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f9e05-b5e7-46e2-9011-d1c2fb26fd00 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:48.808345 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f9e05-b5e7-46e2-9011-d1c2fb26fd00 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:48.808654 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f9e05-b5e7-46e2-9011-d1c2fb26fd00 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:48.809910 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dbe639-c3ea-4710-b6c8-1eb4d34bf970 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:48.833520 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76f9e05-b5e7-46e2-9011-d1c2fb26fd00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:23:48.843188 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-b76f9e05-b5e7-46e2-9011-d1c2fb26fd00 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:23:48.843188 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:23:48.843188 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:23:48.843188 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:23:48.843188 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:23:48.843188 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:23:48.843188 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:23:48.843188 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:23:48.843188 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:23:48.843188 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:23:48.843188 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:23:48.843188 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:23:48.843188 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:23:48.843188 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:23:48.843188 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:23:48.843188 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:23:48.844290 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:23:48.844290 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:23:48.844290 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:23:48.844290 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:23:48.844290 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:23:48.844290 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:23:48.844290 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:23:48.844290 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:23:48.844290 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:23:48.844290 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:23:48.844290 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:23:48.844290 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:23:48.844290 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:23:48.844290 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:23:48.844290 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:23:48.844290 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:23:48.845209 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:23:48.845209 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:23:48.845209 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:23:48.845209 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:23:48.845209 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:23:48.845209 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:23:48.845209 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:23:48.845209 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:23:48.845209 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:23:48.845209 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:23:48.845209 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:23:48.845209 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:23:48.849397 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 142/288] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:23:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:23:48.850258 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f9e05-b5e7-46e2-9011-d1c2fb26fd00 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:48.888576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:48.928757 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:48.938587 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dbe639-c3ea-4710-b6c8-1eb4d34bf970 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:48.974840 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84527adb-f479-41f7-8382-ac5a34e24899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:48.975508 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84527adb-f479-41f7-8382-ac5a34e24899 None admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:48.975783 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84527adb-f479-41f7-8382-ac5a34e24899 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:48.976053 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84527adb-f479-41f7-8382-ac5a34e24899 None admin] PATH_INFO: `/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7/users/6ef5ef2364e842b1bf49b6df0f65d06e/roles/49b52d60e4434f05870a37c49b2e6e6b` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:48.978164 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84527adb-f479-41f7-8382-ac5a34e24899 None admin] RBAC: Authorizing `identity:create_grant(project_id=a7f9d22fd79e4b1fa16a2f60fc6faec7, user_id=6ef5ef2364e842b1bf49b6df0f65d06e, role_id=49b52d60e4434f05870a37c49b2e6e6b)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:48.979616 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84527adb-f479-41f7-8382-ac5a34e24899 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:49.056938 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 147/289] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Aug 24 05:23:48 2022] PUT /identity/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7/users/6ef5ef2364e842b1bf49b6df0f65d06e/roles/49b52d60e4434f05870a37c49b2e6e6b => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:23:49.062907 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f9e05-b5e7-46e2-9011-d1c2fb26fd00 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:49.103770 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94fefa7-386f-4932-92f1-5bd430b71feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:49.104885 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94fefa7-386f-4932-92f1-5bd430b71feb None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:49.105504 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94fefa7-386f-4932-92f1-5bd430b71feb None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:49.105976 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94fefa7-386f-4932-92f1-5bd430b71feb None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:49.106919 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94fefa7-386f-4932-92f1-5bd430b71feb None admin] RBAC: Adding query filter params (name=member) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:49.107425 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94fefa7-386f-4932-92f1-5bd430b71feb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:49.109172 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94fefa7-386f-4932-92f1-5bd430b71feb None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:49.119911 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 143/290] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Aug 24 05:23:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:49.127062 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84527adb-f479-41f7-8382-ac5a34e24899 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:49.255647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84527adb-f479-41f7-8382-ac5a34e24899 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:49.270319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:49.293908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58132ccb-5017-4b6c-b5d1-7b0545b6bb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:49.294557 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58132ccb-5017-4b6c-b5d1-7b0545b6bb2c None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:49.294891 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58132ccb-5017-4b6c-b5d1-7b0545b6bb2c None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:49.295170 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58132ccb-5017-4b6c-b5d1-7b0545b6bb2c None admin] PATH_INFO: `/v3/users/2117f4e51a7d4a7f9295f2f8c12e8713` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:49.296312 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58132ccb-5017-4b6c-b5d1-7b0545b6bb2c None admin] RBAC: Authorizing `identity:get_user(user_id=2117f4e51a7d4a7f9295f2f8c12e8713)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:49.298283 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58132ccb-5017-4b6c-b5d1-7b0545b6bb2c None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:49.299948 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 148/291] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:23:49 2022] GET /identity/v3/users/2117f4e51a7d4a7f9295f2f8c12e8713 => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:23:49.305748 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94fefa7-386f-4932-92f1-5bd430b71feb None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:49.400790 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:49.417318 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94fefa7-386f-4932-92f1-5bd430b71feb None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:49.452646 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8be5c3-cc25-4a0f-98da-26d239a3732e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:49.453214 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8be5c3-cc25-4a0f-98da-26d239a3732e None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:49.454190 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8be5c3-cc25-4a0f-98da-26d239a3732e None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:49.454518 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8be5c3-cc25-4a0f-98da-26d239a3732e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:49.529809 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8be5c3-cc25-4a0f-98da-26d239a3732e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:49.531628 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8be5c3-cc25-4a0f-98da-26d239a3732e None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:49.628791 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:49.635602 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 144/292] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:23:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:23:49.641931 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58132ccb-5017-4b6c-b5d1-7b0545b6bb2c None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:49.752857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58132ccb-5017-4b6c-b5d1-7b0545b6bb2c None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:49.792988 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc233d9-b9a1-426c-b83d-44e8aa0363ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:49.793722 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc233d9-b9a1-426c-b83d-44e8aa0363ae None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:49.793902 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc233d9-b9a1-426c-b83d-44e8aa0363ae None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:49.794177 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc233d9-b9a1-426c-b83d-44e8aa0363ae None admin] PATH_INFO: `/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:49.795369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc233d9-b9a1-426c-b83d-44e8aa0363ae None admin] RBAC: Authorizing `identity:get_project(project_id=a7f9d22fd79e4b1fa16a2f60fc6faec7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:49.796963 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc233d9-b9a1-426c-b83d-44e8aa0363ae None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:49.799640 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 149/293] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:23:49 2022] GET /identity/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:49.805572 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8be5c3-cc25-4a0f-98da-26d239a3732e None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:49.913201 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8be5c3-cc25-4a0f-98da-26d239a3732e None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:49.955471 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3739d-1bca-4deb-8824-9bcbad0d210e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:49.956079 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3739d-1bca-4deb-8824-9bcbad0d210e None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:49.956445 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3739d-1bca-4deb-8824-9bcbad0d210e None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:49.956730 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3739d-1bca-4deb-8824-9bcbad0d210e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:49.957517 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf3739d-1bca-4deb-8824-9bcbad0d210e None admin] RBAC: Adding query filter params (role.id=87862cd6eaeb4d5fae089a801aac4f87, scope.project.id=a7f9d22fd79e4b1fa16a2f60fc6faec7, user.id=2117f4e51a7d4a7f9295f2f8c12e8713) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:49.957831 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf3739d-1bca-4deb-8824-9bcbad0d210e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:49.959983 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf3739d-1bca-4deb-8824-9bcbad0d210e None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:49.966826 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 145/294] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:23:49 2022] GET /identity/v3/role_assignments?user.id=2117f4e51a7d4a7f9295f2f8c12e8713&scope.project.id=a7f9d22fd79e4b1fa16a2f60fc6faec7&role.id=87862cd6eaeb4d5fae089a801aac4f87 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:50.327789 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc0406-f45e-4c0c-9da0-54454a9cbb40 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:50.328208 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc0406-f45e-4c0c-9da0-54454a9cbb40 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:50.329091 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc0406-f45e-4c0c-9da0-54454a9cbb40 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:50.330846 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 150/295] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:50.336635 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a8b96-c2d3-4117-8823-433897762d6f None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:50.336923 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a8b96-c2d3-4117-8823-433897762d6f None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:50.337200 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a8b96-c2d3-4117-8823-433897762d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:50.358313 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-f75a8b96-c2d3-4117-8823-433897762d6f None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:50.359708 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-f75a8b96-c2d3-4117-8823-433897762d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:50.386000 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700f21d-8cd9-4d99-8554-65571cedbaa6 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:50.386606 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700f21d-8cd9-4d99-8554-65571cedbaa6 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:50.387080 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700f21d-8cd9-4d99-8554-65571cedbaa6 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:50.388932 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 151/296] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:50.396768 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c7821-9e10-4826-b2a7-a13b5bbfa459 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:50.397362 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c7821-9e10-4826-b2a7-a13b5bbfa459 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:50.397832 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c7821-9e10-4826-b2a7-a13b5bbfa459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:50.410191 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:50.429321 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-154c7821-9e10-4826-b2a7-a13b5bbfa459 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:50.432532 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-154c7821-9e10-4826-b2a7-a13b5bbfa459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:50.448895 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 146/297] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:50.499503 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:50.527083 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 152/298] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:23:50.738355 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a8b96-c2d3-4117-8823-433897762d6f None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:50.806642 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c7821-9e10-4826-b2a7-a13b5bbfa459 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:50.851816 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a8b96-c2d3-4117-8823-433897762d6f None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:50.890063 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112215d4-60e2-4c9a-97c1-8b8092e8cda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:50.892219 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112215d4-60e2-4c9a-97c1-8b8092e8cda3 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:50.892540 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112215d4-60e2-4c9a-97c1-8b8092e8cda3 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:50.892822 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112215d4-60e2-4c9a-97c1-8b8092e8cda3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:50.893951 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112215d4-60e2-4c9a-97c1-8b8092e8cda3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:50.895409 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112215d4-60e2-4c9a-97c1-8b8092e8cda3 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:50.897175 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 147/299] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:23:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:50.898157 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca87b54-50d6-432d-a972-3cc3404ee09b None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:50.898449 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca87b54-50d6-432d-a972-3cc3404ee09b None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:50.898730 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca87b54-50d6-432d-a972-3cc3404ee09b None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:50.900029 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 148/300] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:50.904669 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca87b54-50d6-432d-a972-3cc3404ee09b None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:50.929576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c7821-9e10-4826-b2a7-a13b5bbfa459 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:50.955668 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:50.970250 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce5e05a-16b0-4676-a8d4-86ee2f6edc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:50.970972 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5e05a-16b0-4676-a8d4-86ee2f6edc57 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:50.971155 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5e05a-16b0-4676-a8d4-86ee2f6edc57 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:50.971425 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5e05a-16b0-4676-a8d4-86ee2f6edc57 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:50.983002 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce5e05a-16b0-4676-a8d4-86ee2f6edc57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:23:50.989782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-8ce5e05a-16b0-4676-a8d4-86ee2f6edc57 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:23:50.989782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:23:50.989782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:23:50.989782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:23:50.989782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:23:50.989782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:23:50.989782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:23:50.989782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:23:50.989782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:23:50.989782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:23:50.989782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:23:50.989782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:23:50.989782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:23:50.989782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:23:50.989782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:23:50.989782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:23:50.990808 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:23:50.990808 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:23:50.990808 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:23:50.990808 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:23:50.990808 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:23:50.990808 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:23:50.990808 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:23:50.990808 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:23:50.990808 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:23:50.990808 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:23:50.990808 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:23:50.990808 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:23:50.990808 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:23:50.990808 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:23:50.990808 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:23:50.990808 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:23:50.991773 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:23:50.991773 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:23:50.991773 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:23:50.991773 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:23:50.991773 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:23:50.991773 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:23:50.991773 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:23:50.991773 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:23:50.991773 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:23:50.991773 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:23:50.991773 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:23:50.991773 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:23:50.991773 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 153/301] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:23:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:23:50.992602 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367b23b-f66b-40fc-926b-d83f42096ac2 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:50.992602 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367b23b-f66b-40fc-926b-d83f42096ac2 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:50.992602 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367b23b-f66b-40fc-926b-d83f42096ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:51.012115 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-e367b23b-f66b-40fc-926b-d83f42096ac2 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:51.014054 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-e367b23b-f66b-40fc-926b-d83f42096ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:51.024534 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca87b54-50d6-432d-a972-3cc3404ee09b None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:51.045336 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:51.065094 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e5522b-c02e-4a13-9ecb-1b6d8fa7cf35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:51.067708 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e5522b-c02e-4a13-9ecb-1b6d8fa7cf35 None admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:51.067708 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e5522b-c02e-4a13-9ecb-1b6d8fa7cf35 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:51.067708 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e5522b-c02e-4a13-9ecb-1b6d8fa7cf35 None admin] PATH_INFO: `/v3/groups` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:51.067708 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e5522b-c02e-4a13-9ecb-1b6d8fa7cf35 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:51.068354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e5522b-c02e-4a13-9ecb-1b6d8fa7cf35 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:51.084920 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 154/302] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:51.096345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e367b23b-f66b-40fc-926b-d83f42096ac2 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:51.183747 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 149/303] 10.222.0.15 () {60 vars in 1134 bytes} [Wed Aug 24 05:23:50 2022] POST /identity/v3/groups => generated 229 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:23:51.202571 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e367b23b-f66b-40fc-926b-d83f42096ac2 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:51.243248 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd9847-edb9-48ee-a980-6c3b8d35e633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:51.243800 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd9847-edb9-48ee-a980-6c3b8d35e633 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:51.244083 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd9847-edb9-48ee-a980-6c3b8d35e633 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:51.244424 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd9847-edb9-48ee-a980-6c3b8d35e633 None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:51.245137 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbd9847-edb9-48ee-a980-6c3b8d35e633 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:51.245450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbd9847-edb9-48ee-a980-6c3b8d35e633 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:51.247012 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbd9847-edb9-48ee-a980-6c3b8d35e633 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:51.257787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 155/304] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Aug 24 05:23:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:51.263576 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e5522b-c02e-4a13-9ecb-1b6d8fa7cf35 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:51.372644 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd9847-edb9-48ee-a980-6c3b8d35e633 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:51.381737 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e5522b-c02e-4a13-9ecb-1b6d8fa7cf35 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:51.383343 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:51.419097 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:51.433561 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6342a51-5a1b-449b-9aae-9b520d11e6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:51.434780 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6342a51-5a1b-449b-9aae-9b520d11e6e3 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:51.435359 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6342a51-5a1b-449b-9aae-9b520d11e6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:51.436058 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6342a51-5a1b-449b-9aae-9b520d11e6e3 None admin] PATH_INFO: `/v3/users/adbcbc85ba344aeb9e669954f6c63f3c` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:51.464493 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6342a51-5a1b-449b-9aae-9b520d11e6e3 None admin] RBAC: Authorizing `identity:get_user(user_id=adbcbc85ba344aeb9e669954f6c63f3c)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:51.466135 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6342a51-5a1b-449b-9aae-9b520d11e6e3 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:51.468700 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 150/305] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:23:51 2022] GET /identity/v3/users/adbcbc85ba344aeb9e669954f6c63f3c => generated 293 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:51.470039 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11619e7-1ebc-49ba-b022-adb8fbd50244 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:51.470328 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11619e7-1ebc-49ba-b022-adb8fbd50244 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:51.470661 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11619e7-1ebc-49ba-b022-adb8fbd50244 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:51.472123 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 151/306] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:51.475020 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11619e7-1ebc-49ba-b022-adb8fbd50244 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:51.499377 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd9847-edb9-48ee-a980-6c3b8d35e633 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:51.539118 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0737ae3-3c4c-41aa-9bb7-b04772486b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:51.539756 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0737ae3-3c4c-41aa-9bb7-b04772486b9a None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:51.540047 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0737ae3-3c4c-41aa-9bb7-b04772486b9a None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:51.540321 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0737ae3-3c4c-41aa-9bb7-b04772486b9a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:51.553018 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0737ae3-3c4c-41aa-9bb7-b04772486b9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:23:51.558316 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-d0737ae3-3c4c-41aa-9bb7-b04772486b9a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:23:51.558316 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:23:51.558316 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:23:51.558316 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:23:51.558316 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:23:51.558316 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:23:51.558316 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:23:51.558316 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:23:51.558316 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:23:51.558316 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:23:51.558316 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:23:51.558316 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:23:51.558316 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:23:51.558316 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:23:51.558316 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:23:51.558316 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:23:51.559197 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:23:51.559197 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:23:51.559197 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:23:51.559197 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:23:51.559197 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:23:51.559197 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:23:51.559197 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:23:51.559197 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:23:51.559197 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:23:51.559197 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:23:51.559197 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:23:51.559197 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:23:51.559197 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:23:51.559197 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:23:51.559197 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:23:51.559197 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:23:51.560041 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:23:51.560041 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:23:51.560041 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:23:51.560041 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:23:51.560041 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:23:51.560041 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:23:51.560041 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:23:51.560041 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:23:51.560041 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:23:51.560041 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:23:51.560041 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:23:51.560041 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:23:51.564560 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 156/307] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:23:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:23:51.566015 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e17e3-f5b2-4182-9501-00bacc048f20 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:51.566301 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e17e3-f5b2-4182-9501-00bacc048f20 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:51.566574 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e17e3-f5b2-4182-9501-00bacc048f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:51.584519 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-895e17e3-f5b2-4182-9501-00bacc048f20 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:51.585716 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-895e17e3-f5b2-4182-9501-00bacc048f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:51.595785 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11619e7-1ebc-49ba-b022-adb8fbd50244 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:51.634789 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941ee457-f5d6-44c8-9df6-77ed3f03f1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:51.635753 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ee457-f5d6-44c8-9df6-77ed3f03f1a1 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:51.635753 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ee457-f5d6-44c8-9df6-77ed3f03f1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:51.636031 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ee457-f5d6-44c8-9df6-77ed3f03f1a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:51.636775 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941ee457-f5d6-44c8-9df6-77ed3f03f1a1 None admin] RBAC: Adding query filter params (role.id=49b52d60e4434f05870a37c49b2e6e6b, user.id=adbcbc85ba344aeb9e669954f6c63f3c, scope.system=all) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:51.637086 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941ee457-f5d6-44c8-9df6-77ed3f03f1a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:51.638511 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941ee457-f5d6-44c8-9df6-77ed3f03f1a1 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:51.652423 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 152/308] 10.222.0.15 () {56 vars in 1308 bytes} [Wed Aug 24 05:23:51 2022] GET /identity/v3/role_assignments?user.id=adbcbc85ba344aeb9e669954f6c63f3c&scope.system=all&role.id=49b52d60e4434f05870a37c49b2e6e6b => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:51.652423 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941ee457-f5d6-44c8-9df6-77ed3f03f1a1 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:51.663080 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 157/309] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:51.682617 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:51.769439 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941ee457-f5d6-44c8-9df6-77ed3f03f1a1 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:51.816283 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac6f8f-1925-4342-8ac3-b894d1549ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:51.817354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac6f8f-1925-4342-8ac3-b894d1549ac4 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:51.817863 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac6f8f-1925-4342-8ac3-b894d1549ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:51.818341 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac6f8f-1925-4342-8ac3-b894d1549ac4 None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:51.819246 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac6f8f-1925-4342-8ac3-b894d1549ac4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:51.819752 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac6f8f-1925-4342-8ac3-b894d1549ac4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:51.821371 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac6f8f-1925-4342-8ac3-b894d1549ac4 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:51.832240 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 153/310] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Aug 24 05:23:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:51.838026 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e17e3-f5b2-4182-9501-00bacc048f20 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:51.866968 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:51.955081 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e17e3-f5b2-4182-9501-00bacc048f20 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:51.972322 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac6f8f-1925-4342-8ac3-b894d1549ac4 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:51.993839 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed05802-50b0-475c-ba5a-8b6a9a8fda37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:51.995062 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed05802-50b0-475c-ba5a-8b6a9a8fda37 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:51.995518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed05802-50b0-475c-ba5a-8b6a9a8fda37 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:51.996229 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed05802-50b0-475c-ba5a-8b6a9a8fda37 None admin] PATH_INFO: `/v3/users/0f2c9f2165a04d738e90d3a7f7deb767` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:51.997971 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed05802-50b0-475c-ba5a-8b6a9a8fda37 None admin] RBAC: Authorizing `identity:get_user(user_id=0f2c9f2165a04d738e90d3a7f7deb767)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:51.999592 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed05802-50b0-475c-ba5a-8b6a9a8fda37 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:52.003158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 158/311] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:23:51 2022] GET /identity/v3/users/0f2c9f2165a04d738e90d3a7f7deb767 => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:52.009001 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed05802-50b0-475c-ba5a-8b6a9a8fda37 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:52.085761 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:52.103791 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac6f8f-1925-4342-8ac3-b894d1549ac4 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:52.130208 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:52.147095 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed05802-50b0-475c-ba5a-8b6a9a8fda37 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:52.148605 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5853983-6472-4d2f-9c7c-0fe6845a0451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:52.149520 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5853983-6472-4d2f-9c7c-0fe6845a0451 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:52.149899 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5853983-6472-4d2f-9c7c-0fe6845a0451 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:52.151096 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5853983-6472-4d2f-9c7c-0fe6845a0451 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:52.165453 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5853983-6472-4d2f-9c7c-0fe6845a0451 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:23:52.174779 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-d5853983-6472-4d2f-9c7c-0fe6845a0451 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:23:52.174779 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:23:52.174779 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:23:52.174779 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:23:52.174779 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:23:52.174779 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:23:52.174779 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:23:52.174779 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:23:52.174779 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:23:52.174779 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:23:52.174779 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:23:52.174779 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:23:52.174779 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:23:52.174779 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:23:52.174779 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:23:52.174779 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:23:52.175931 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:23:52.175931 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:23:52.175931 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:23:52.175931 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:23:52.175931 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:23:52.175931 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:23:52.175931 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:23:52.175931 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:23:52.175931 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:23:52.175931 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:23:52.175931 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:23:52.175931 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:23:52.175931 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:23:52.175931 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:23:52.175931 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:23:52.175931 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:23:52.176942 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:23:52.176942 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:23:52.176942 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:23:52.176942 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:23:52.176942 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:23:52.176942 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:23:52.176942 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:23:52.176942 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:23:52.176942 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:23:52.176942 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:23:52.176942 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:23:52.176942 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:23:52.176942 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 154/312] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:23:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:23:52.184410 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5853983-6472-4d2f-9c7c-0fe6845a0451 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:52.199432 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7008f152-d8cd-469d-9057-1894edaca74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:52.200103 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008f152-d8cd-469d-9057-1894edaca74a None admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:52.200391 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008f152-d8cd-469d-9057-1894edaca74a None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:52.200694 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008f152-d8cd-469d-9057-1894edaca74a None admin] PATH_INFO: `/v3/system/users/0f2c9f2165a04d738e90d3a7f7deb767/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:52.202186 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7008f152-d8cd-469d-9057-1894edaca74a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0f2c9f2165a04d738e90d3a7f7deb767, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:52.204798 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7008f152-d8cd-469d-9057-1894edaca74a None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:52.316881 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 159/313] 10.222.0.15 () {58 vars in 1334 bytes} [Wed Aug 24 05:23:52 2022] PUT /identity/v3/system/users/0f2c9f2165a04d738e90d3a7f7deb767/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:23:52.326693 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5853983-6472-4d2f-9c7c-0fe6845a0451 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:52.377407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634eab0b-b13b-4ba3-8d92-7044d2592d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:52.378333 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634eab0b-b13b-4ba3-8d92-7044d2592d68 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:52.378807 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634eab0b-b13b-4ba3-8d92-7044d2592d68 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:52.379309 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634eab0b-b13b-4ba3-8d92-7044d2592d68 None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:52.380167 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634eab0b-b13b-4ba3-8d92-7044d2592d68 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:52.380698 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634eab0b-b13b-4ba3-8d92-7044d2592d68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:52.382644 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634eab0b-b13b-4ba3-8d92-7044d2592d68 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:52.404828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 155/314] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Aug 24 05:23:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:52.407519 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7008f152-d8cd-469d-9057-1894edaca74a None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:52.524316 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7008f152-d8cd-469d-9057-1894edaca74a None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:52.562180 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ed88d-8d35-43da-9da6-d4768ee32bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:52.562933 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ed88d-8d35-43da-9da6-d4768ee32bfa None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:52.563234 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ed88d-8d35-43da-9da6-d4768ee32bfa None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:52.563610 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ed88d-8d35-43da-9da6-d4768ee32bfa None admin] PATH_INFO: `/v3/users/6ef5ef2364e842b1bf49b6df0f65d06e` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:52.564807 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96ed88d-8d35-43da-9da6-d4768ee32bfa None admin] RBAC: Authorizing `identity:get_user(user_id=6ef5ef2364e842b1bf49b6df0f65d06e)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:52.566298 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96ed88d-8d35-43da-9da6-d4768ee32bfa None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:52.568260 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 160/315] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:23:52 2022] GET /identity/v3/users/6ef5ef2364e842b1bf49b6df0f65d06e => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:52.574516 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634eab0b-b13b-4ba3-8d92-7044d2592d68 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:52.588348 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:52.683998 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634eab0b-b13b-4ba3-8d92-7044d2592d68 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:52.724131 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b23ca-f15d-4943-b0ee-c2cdf45aacd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:52.724784 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b23ca-f15d-4943-b0ee-c2cdf45aacd5 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:52.725070 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b23ca-f15d-4943-b0ee-c2cdf45aacd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:52.725348 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b23ca-f15d-4943-b0ee-c2cdf45aacd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:52.802336 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605b23ca-f15d-4943-b0ee-c2cdf45aacd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:52.803797 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605b23ca-f15d-4943-b0ee-c2cdf45aacd5 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:52.815373 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:52.916154 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 156/316] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:23:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:23:52.921615 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ed88d-8d35-43da-9da6-d4768ee32bfa None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:52.944822 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:53.027164 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ed88d-8d35-43da-9da6-d4768ee32bfa None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:53.063203 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14655e6-d033-4149-b7d5-f63037fde058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:53.064109 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14655e6-d033-4149-b7d5-f63037fde058 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:53.064570 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14655e6-d033-4149-b7d5-f63037fde058 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:53.065010 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14655e6-d033-4149-b7d5-f63037fde058 None admin] PATH_INFO: `/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:53.066294 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14655e6-d033-4149-b7d5-f63037fde058 None admin] RBAC: Authorizing `identity:get_project(project_id=a7f9d22fd79e4b1fa16a2f60fc6faec7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:53.067890 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14655e6-d033-4149-b7d5-f63037fde058 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:53.069729 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 161/317] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:23:52 2022] GET /identity/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:53.077932 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b23ca-f15d-4943-b0ee-c2cdf45aacd5 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:53.184573 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b23ca-f15d-4943-b0ee-c2cdf45aacd5 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:53.218543 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:53.222525 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b177aba-3842-49c8-a288-312607db3925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:53.223318 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b177aba-3842-49c8-a288-312607db3925 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:53.223745 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b177aba-3842-49c8-a288-312607db3925 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:53.224176 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b177aba-3842-49c8-a288-312607db3925 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:53.225080 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b177aba-3842-49c8-a288-312607db3925 None admin] RBAC: Adding query filter params (role.id=49b52d60e4434f05870a37c49b2e6e6b, scope.project.id=a7f9d22fd79e4b1fa16a2f60fc6faec7, user.id=6ef5ef2364e842b1bf49b6df0f65d06e) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:53.225541 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b177aba-3842-49c8-a288-312607db3925 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:53.227183 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b177aba-3842-49c8-a288-312607db3925 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:53.233733 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 157/318] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:23:53 2022] GET /identity/v3/role_assignments?user.id=6ef5ef2364e842b1bf49b6df0f65d06e&scope.project.id=a7f9d22fd79e4b1fa16a2f60fc6faec7&role.id=49b52d60e4434f05870a37c49b2e6e6b => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:53.324118 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0dc95-b5c4-43b6-ac50-d5c13b020ce2 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:53.324427 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0dc95-b5c4-43b6-ac50-d5c13b020ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:53.324726 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0dc95-b5c4-43b6-ac50-d5c13b020ce2 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:53.326090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 162/319] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:53.332992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0cd114-9826-4bc4-b37c-6d6eda3292b3 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:53.333265 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0cd114-9826-4bc4-b37c-6d6eda3292b3 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:53.333546 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0cd114-9826-4bc4-b37c-6d6eda3292b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:53.353612 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-ac0cd114-9826-4bc4-b37c-6d6eda3292b3 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:53.354830 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-ac0cd114-9826-4bc4-b37c-6d6eda3292b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:53.431363 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 158/320] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:53.708152 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a0dc95-b5c4-43b6-ac50-d5c13b020ce2 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:53.768229 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0585534-3b53-40d7-9c07-2893bd51e6e6 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:53.768665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0585534-3b53-40d7-9c07-2893bd51e6e6 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:53.768882 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0585534-3b53-40d7-9c07-2893bd51e6e6 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:53.770445 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 159/321] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:53.775505 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04412b08-85ab-4fb7-9203-9a705fff8a0c None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:53.775866 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04412b08-85ab-4fb7-9203-9a705fff8a0c None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:53.776219 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04412b08-85ab-4fb7-9203-9a705fff8a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:53.856012 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-04412b08-85ab-4fb7-9203-9a705fff8a0c None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:53.857272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-04412b08-85ab-4fb7-9203-9a705fff8a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:53.867011 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:53.880988 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a0dc95-b5c4-43b6-ac50-d5c13b020ce2 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:53.919712 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9744ef8-d68c-496b-aaa6-154556ba1197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:53.920351 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9744ef8-d68c-496b-aaa6-154556ba1197 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:53.920683 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9744ef8-d68c-496b-aaa6-154556ba1197 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:53.920960 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9744ef8-d68c-496b-aaa6-154556ba1197 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:53.922815 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9744ef8-d68c-496b-aaa6-154556ba1197 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:53.924542 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9744ef8-d68c-496b-aaa6-154556ba1197 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:53.926129 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 163/322] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:23:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:53.931498 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9744ef8-d68c-496b-aaa6-154556ba1197 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:53.932364 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 160/323] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:23:54.037479 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9744ef8-d68c-496b-aaa6-154556ba1197 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:54.073300 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a909e36-f4da-4347-90ba-68f88e5709c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:54.073913 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a909e36-f4da-4347-90ba-68f88e5709c9 None admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:54.074193 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a909e36-f4da-4347-90ba-68f88e5709c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:54.074466 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a909e36-f4da-4347-90ba-68f88e5709c9 None admin] PATH_INFO: `/v3/groups` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:54.075092 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a909e36-f4da-4347-90ba-68f88e5709c9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:54.076550 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a909e36-f4da-4347-90ba-68f88e5709c9 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:54.192428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 164/324] 10.222.0.15 () {60 vars in 1134 bytes} [Wed Aug 24 05:23:53 2022] POST /identity/v3/groups => generated 226 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:23:54.198213 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04412b08-85ab-4fb7-9203-9a705fff8a0c None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:54.234073 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:54.316307 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04412b08-85ab-4fb7-9203-9a705fff8a0c None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:54.354800 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84537c57-67c7-47fc-8ca7-6921ca57f550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:54.355439 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84537c57-67c7-47fc-8ca7-6921ca57f550 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:54.355720 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84537c57-67c7-47fc-8ca7-6921ca57f550 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:54.355992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84537c57-67c7-47fc-8ca7-6921ca57f550 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:54.367363 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84537c57-67c7-47fc-8ca7-6921ca57f550 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:23:54.372133 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-84537c57-67c7-47fc-8ca7-6921ca57f550 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:23:54.372133 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:23:54.372133 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:23:54.372133 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:23:54.372133 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:23:54.372133 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:23:54.372133 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:23:54.372133 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:23:54.372133 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:23:54.372133 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:23:54.372133 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:23:54.372133 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:23:54.372133 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:23:54.372133 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:23:54.372133 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:23:54.372133 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:23:54.373087 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:23:54.373087 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:23:54.373087 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:23:54.373087 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:23:54.373087 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:23:54.373087 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:23:54.373087 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:23:54.373087 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:23:54.373087 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:23:54.373087 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:23:54.373087 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:23:54.373087 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:23:54.373087 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:23:54.373087 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:23:54.373087 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:23:54.373087 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:23:54.373944 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:23:54.373944 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:23:54.373944 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:23:54.373944 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:23:54.373944 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:23:54.373944 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:23:54.373944 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:23:54.373944 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:23:54.373944 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:23:54.373944 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:23:54.373944 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:23:54.373944 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:23:54.373944 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 161/325] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:23:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:23:54.379567 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a909e36-f4da-4347-90ba-68f88e5709c9 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:54.403212 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:54.440130 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ffc46-e88f-498e-bf4f-0e203eb9f793 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:54.441807 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ffc46-e88f-498e-bf4f-0e203eb9f793 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:54.444106 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ffc46-e88f-498e-bf4f-0e203eb9f793 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:54.446486 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 162/326] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:54.452380 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965f3661-74f2-4981-b157-dfa391f1ff22 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:54.452702 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965f3661-74f2-4981-b157-dfa391f1ff22 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:54.454058 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965f3661-74f2-4981-b157-dfa391f1ff22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:54.475023 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-965f3661-74f2-4981-b157-dfa391f1ff22 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:54.476575 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-965f3661-74f2-4981-b157-dfa391f1ff22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:54.498693 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a909e36-f4da-4347-90ba-68f88e5709c9 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:54.543098 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70713b62-57f3-495d-8623-f3f944eccf8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:54.543941 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70713b62-57f3-495d-8623-f3f944eccf8d None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:54.544282 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70713b62-57f3-495d-8623-f3f944eccf8d None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:54.544704 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70713b62-57f3-495d-8623-f3f944eccf8d None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:54.545421 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70713b62-57f3-495d-8623-f3f944eccf8d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:54.545742 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70713b62-57f3-495d-8623-f3f944eccf8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:54.547921 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70713b62-57f3-495d-8623-f3f944eccf8d None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:54.561725 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 165/327] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Aug 24 05:23:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:54.564222 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70713b62-57f3-495d-8623-f3f944eccf8d None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:54.567874 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 163/328] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:23:54.657441 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:54.680969 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70713b62-57f3-495d-8623-f3f944eccf8d None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:54.720474 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f87a283-2d7f-44b2-84e8-58f0dda31b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:54.721154 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87a283-2d7f-44b2-84e8-58f0dda31b36 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:54.721437 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87a283-2d7f-44b2-84e8-58f0dda31b36 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:54.721711 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87a283-2d7f-44b2-84e8-58f0dda31b36 None admin] PATH_INFO: `/v3/users/adbcbc85ba344aeb9e669954f6c63f3c` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:54.722901 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f87a283-2d7f-44b2-84e8-58f0dda31b36 None admin] RBAC: Authorizing `identity:get_user(user_id=adbcbc85ba344aeb9e669954f6c63f3c)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:54.724356 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f87a283-2d7f-44b2-84e8-58f0dda31b36 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:54.726273 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 166/329] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:23:54 2022] GET /identity/v3/users/adbcbc85ba344aeb9e669954f6c63f3c => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:54.731897 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965f3661-74f2-4981-b157-dfa391f1ff22 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:54.764162 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:54.865974 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f87a283-2d7f-44b2-84e8-58f0dda31b36 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:54.956308 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965f3661-74f2-4981-b157-dfa391f1ff22 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:54.997993 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f87a283-2d7f-44b2-84e8-58f0dda31b36 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:55.024641 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49772962-387c-424e-8a2e-ca4272251a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:55.025372 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49772962-387c-424e-8a2e-ca4272251a84 None admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:55.025662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49772962-387c-424e-8a2e-ca4272251a84 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:55.025943 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49772962-387c-424e-8a2e-ca4272251a84 None admin] PATH_INFO: `/v3/system/users/adbcbc85ba344aeb9e669954f6c63f3c/roles/49b52d60e4434f05870a37c49b2e6e6b` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:55.028210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49772962-387c-424e-8a2e-ca4272251a84 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=adbcbc85ba344aeb9e669954f6c63f3c, role_id=49b52d60e4434f05870a37c49b2e6e6b)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:55.029695 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49772962-387c-424e-8a2e-ca4272251a84 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:55.055836 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b13ee-e128-4edf-a265-866685250cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:55.056499 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b13ee-e128-4edf-a265-866685250cd6 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:55.056782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b13ee-e128-4edf-a265-866685250cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:55.057052 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b13ee-e128-4edf-a265-866685250cd6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:55.073076 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07b13ee-e128-4edf-a265-866685250cd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:23:55.078710 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-e07b13ee-e128-4edf-a265-866685250cd6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:23:55.078710 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:23:55.078710 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:23:55.078710 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:23:55.078710 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:23:55.078710 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:23:55.078710 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:23:55.078710 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:23:55.078710 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:23:55.078710 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:23:55.078710 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:23:55.078710 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:23:55.078710 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:23:55.078710 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:23:55.078710 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:23:55.078710 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:23:55.079724 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:23:55.079724 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:23:55.079724 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:23:55.079724 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:23:55.079724 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:23:55.079724 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:23:55.079724 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:23:55.079724 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:23:55.079724 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:23:55.079724 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:23:55.079724 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:23:55.079724 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:23:55.079724 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:23:55.079724 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:23:55.079724 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:23:55.079724 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:23:55.080537 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:23:55.080537 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:23:55.080537 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:23:55.080537 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:23:55.080537 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:23:55.080537 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:23:55.080537 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:23:55.080537 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:23:55.080537 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:23:55.080537 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:23:55.080537 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:23:55.080537 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:23:55.084589 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 167/330] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:23:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:23:55.086064 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b13ee-e128-4edf-a265-866685250cd6 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:55.096623 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:55.103910 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 164/331] 10.222.0.15 () {58 vars in 1334 bytes} [Wed Aug 24 05:23:54 2022] PUT /identity/v3/system/users/adbcbc85ba344aeb9e669954f6c63f3c/roles/49b52d60e4434f05870a37c49b2e6e6b => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:23:55.217355 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b13ee-e128-4edf-a265-866685250cd6 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:55.265507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7fe0d6-f2a0-4ada-ab3f-698bdf12ca54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:55.266731 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7fe0d6-f2a0-4ada-ab3f-698bdf12ca54 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:55.267011 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7fe0d6-f2a0-4ada-ab3f-698bdf12ca54 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:55.267379 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7fe0d6-f2a0-4ada-ab3f-698bdf12ca54 None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:55.268150 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7fe0d6-f2a0-4ada-ab3f-698bdf12ca54 None admin] RBAC: Adding query filter params (name=member) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:55.268481 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7fe0d6-f2a0-4ada-ab3f-698bdf12ca54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:55.269916 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7fe0d6-f2a0-4ada-ab3f-698bdf12ca54 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:55.280855 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 168/332] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Aug 24 05:23:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:23:55.287394 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49772962-387c-424e-8a2e-ca4272251a84 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:55.395854 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:55.415492 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49772962-387c-424e-8a2e-ca4272251a84 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:55.459311 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca4fbf5-4f23-40dd-b273-c4daebfaed6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:55.460330 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca4fbf5-4f23-40dd-b273-c4daebfaed6d None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:55.460843 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca4fbf5-4f23-40dd-b273-c4daebfaed6d None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:55.461376 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca4fbf5-4f23-40dd-b273-c4daebfaed6d None admin] PATH_INFO: `/v3/users/0f2c9f2165a04d738e90d3a7f7deb767` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:55.462777 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca4fbf5-4f23-40dd-b273-c4daebfaed6d None admin] RBAC: Authorizing `identity:get_user(user_id=0f2c9f2165a04d738e90d3a7f7deb767)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:55.464454 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca4fbf5-4f23-40dd-b273-c4daebfaed6d None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:55.467543 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 165/333] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:23:55 2022] GET /identity/v3/users/0f2c9f2165a04d738e90d3a7f7deb767 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:55.473906 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7fe0d6-f2a0-4ada-ab3f-698bdf12ca54 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:55.564108 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:55.613745 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7fe0d6-f2a0-4ada-ab3f-698bdf12ca54 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:55.653223 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2f5a2c-9b36-4980-9b8e-34cdae1ccecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:55.654193 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f5a2c-9b36-4980-9b8e-34cdae1ccecc None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:55.654714 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f5a2c-9b36-4980-9b8e-34cdae1ccecc None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:55.655242 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f5a2c-9b36-4980-9b8e-34cdae1ccecc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:55.657256 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2f5a2c-9b36-4980-9b8e-34cdae1ccecc None admin] RBAC: Adding query filter params (role.id=87862cd6eaeb4d5fae089a801aac4f87, user.id=0f2c9f2165a04d738e90d3a7f7deb767, scope.system=all) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:55.657797 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2f5a2c-9b36-4980-9b8e-34cdae1ccecc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:55.659582 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2f5a2c-9b36-4980-9b8e-34cdae1ccecc None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:55.675617 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 169/334] 10.222.0.15 () {56 vars in 1308 bytes} [Wed Aug 24 05:23:55 2022] GET /identity/v3/role_assignments?user.id=0f2c9f2165a04d738e90d3a7f7deb767&scope.system=all&role.id=87862cd6eaeb4d5fae089a801aac4f87 => generated 510 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:56.844468 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d1eaa-39f6-419b-a954-833a7b3576dc None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:56.844996 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d1eaa-39f6-419b-a954-833a7b3576dc None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:56.844996 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d1eaa-39f6-419b-a954-833a7b3576dc None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:56.846370 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 166/335] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:56.852110 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a2d26-d449-4fbf-b89a-c61d7b107a35 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:56.852565 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a2d26-d449-4fbf-b89a-c61d7b107a35 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:56.852744 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a2d26-d449-4fbf-b89a-c61d7b107a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:56.878288 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-692a2d26-d449-4fbf-b89a-c61d7b107a35 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:56.879687 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-692a2d26-d449-4fbf-b89a-c61d7b107a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:56.987124 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 170/336] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:57.383604 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43de299-5a21-4690-b9fc-919259b04b1c None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:57.383885 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43de299-5a21-4690-b9fc-919259b04b1c None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:57.384445 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43de299-5a21-4690-b9fc-919259b04b1c None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:57.386480 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 167/337] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:57.392332 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff9f4b-41ee-4ab9-8029-fbe4102aa96c None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:57.392909 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff9f4b-41ee-4ab9-8029-fbe4102aa96c None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:57.393447 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff9f4b-41ee-4ab9-8029-fbe4102aa96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:57.417274 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-64ff9f4b-41ee-4ab9-8029-fbe4102aa96c None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:57.418837 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-64ff9f4b-41ee-4ab9-8029-fbe4102aa96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:57.467596 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43de299-5a21-4690-b9fc-919259b04b1c None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:57.493075 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:57.508062 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 171/338] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:57.603381 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6d7fd-035b-45c7-9716-7d52272c0cf7 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:57.604208 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6d7fd-035b-45c7-9716-7d52272c0cf7 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:57.605357 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6d7fd-035b-45c7-9716-7d52272c0cf7 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:57.607568 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 172/339] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:57.631962 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f050f-b1f9-4e5b-8ada-621c43824844 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:57.632527 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f050f-b1f9-4e5b-8ada-621c43824844 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:57.632949 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f050f-b1f9-4e5b-8ada-621c43824844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:57.668441 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43de299-5a21-4690-b9fc-919259b04b1c None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:57.677076 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-a85f050f-b1f9-4e5b-8ada-621c43824844 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:57.678500 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-a85f050f-b1f9-4e5b-8ada-621c43824844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:57.730664 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6431c285-7864-4097-8855-e5881feccc66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:57.731227 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6431c285-7864-4097-8855-e5881feccc66 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:57.731515 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6431c285-7864-4097-8855-e5881feccc66 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:57.731790 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6431c285-7864-4097-8855-e5881feccc66 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:57.748275 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6431c285-7864-4097-8855-e5881feccc66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:23:57.754662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-6431c285-7864-4097-8855-e5881feccc66 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:23:57.754662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:23:57.754662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:23:57.754662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:23:57.754662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:23:57.754662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:23:57.754662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:23:57.754662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:23:57.754662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:23:57.754662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:23:57.754662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:23:57.754662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:23:57.754662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:23:57.754662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:23:57.754662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:23:57.754662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:23:57.755564 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:23:57.755564 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:23:57.755564 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:23:57.755564 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:23:57.755564 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:23:57.755564 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:23:57.755564 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:23:57.755564 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:23:57.755564 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:23:57.755564 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:23:57.755564 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:23:57.755564 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:23:57.755564 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:23:57.755564 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:23:57.755564 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:23:57.755564 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:23:57.756352 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:23:57.756352 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:23:57.756352 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:23:57.756352 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:23:57.756352 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:23:57.756352 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:23:57.756352 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:23:57.756352 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:23:57.756352 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:23:57.756352 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:23:57.756352 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:23:57.756352 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:23:57.756352 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 168/340] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:23:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:23:57.761344 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6431c285-7864-4097-8855-e5881feccc66 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:57.792547 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:57.841252 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 173/341] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:57.908476 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6431c285-7864-4097-8855-e5881feccc66 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:57.931203 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f050f-b1f9-4e5b-8ada-621c43824844 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:57.971360 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa086af9-2883-4cc0-8c58-265fd9dd2b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:57.972116 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa086af9-2883-4cc0-8c58-265fd9dd2b75 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:57.972534 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa086af9-2883-4cc0-8c58-265fd9dd2b75 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:57.972829 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa086af9-2883-4cc0-8c58-265fd9dd2b75 None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:57.973634 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa086af9-2883-4cc0-8c58-265fd9dd2b75 None admin] RBAC: Adding query filter params (name=member) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:57.974142 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa086af9-2883-4cc0-8c58-265fd9dd2b75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:57.975803 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa086af9-2883-4cc0-8c58-265fd9dd2b75 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:57.992606 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 169/342] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Aug 24 05:23:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:23:57.997646 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa086af9-2883-4cc0-8c58-265fd9dd2b75 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:58.089751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f050f-b1f9-4e5b-8ada-621c43824844 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:58.120864 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:58.139870 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc64d75-9a20-4745-9114-09d68586c220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:58.140186 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc64d75-9a20-4745-9114-09d68586c220 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:58.144545 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc64d75-9a20-4745-9114-09d68586c220 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:58.144860 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc64d75-9a20-4745-9114-09d68586c220 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:58.156245 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc64d75-9a20-4745-9114-09d68586c220 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:23:58.157915 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:58.170931 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-ebc64d75-9a20-4745-9114-09d68586c220 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:23:58.170931 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:23:58.170931 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:23:58.170931 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:23:58.170931 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:23:58.170931 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:23:58.170931 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:23:58.170931 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:23:58.170931 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:23:58.170931 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:23:58.170931 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:23:58.170931 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:23:58.170931 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:23:58.170931 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:23:58.170931 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:23:58.170931 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:23:58.172126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:23:58.172126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:23:58.172126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:23:58.172126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:23:58.172126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:23:58.172126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:23:58.172126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:23:58.172126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:23:58.172126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:23:58.172126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:23:58.172126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:23:58.172126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:23:58.172126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:23:58.172126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:23:58.172126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:23:58.172126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:23:58.173466 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:23:58.173466 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:23:58.173466 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:23:58.173466 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:23:58.173466 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:23:58.173466 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:23:58.173466 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:23:58.173466 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:23:58.173466 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:23:58.173466 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:23:58.173466 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:23:58.173466 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:23:58.173466 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 174/343] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Aug 24 05:23:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:23:58.197477 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc64d75-9a20-4745-9114-09d68586c220 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:58.197745 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa086af9-2883-4cc0-8c58-265fd9dd2b75 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:58.258571 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e789ff89-f7a4-44a3-b6ea-80195a8fd134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:58.259277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e789ff89-f7a4-44a3-b6ea-80195a8fd134 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:58.259566 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e789ff89-f7a4-44a3-b6ea-80195a8fd134 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:58.259843 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e789ff89-f7a4-44a3-b6ea-80195a8fd134 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:58.370006 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e789ff89-f7a4-44a3-b6ea-80195a8fd134 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:58.371481 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e789ff89-f7a4-44a3-b6ea-80195a8fd134 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:58.383472 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc64d75-9a20-4745-9114-09d68586c220 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:58.425255 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb2aae7-4e0f-4f41-be74-6f8e2ea36821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:58.425884 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2aae7-4e0f-4f41-be74-6f8e2ea36821 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:58.426163 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2aae7-4e0f-4f41-be74-6f8e2ea36821 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:58.426623 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2aae7-4e0f-4f41-be74-6f8e2ea36821 None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:58.428954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb2aae7-4e0f-4f41-be74-6f8e2ea36821 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:58.429473 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb2aae7-4e0f-4f41-be74-6f8e2ea36821 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:58.431077 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb2aae7-4e0f-4f41-be74-6f8e2ea36821 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:58.442382 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 175/344] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:23:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:58.475246 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb2aae7-4e0f-4f41-be74-6f8e2ea36821 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:58.492787 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:58.522142 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 170/345] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:23:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 05:23:58.523471 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92295e4-8d97-4e74-b0aa-f2313306bc9a None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:58.523762 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92295e4-8d97-4e74-b0aa-f2313306bc9a None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:58.524041 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92295e4-8d97-4e74-b0aa-f2313306bc9a None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:58.525556 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 171/346] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:58.527129 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92295e4-8d97-4e74-b0aa-f2313306bc9a None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:58.536575 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:58.621452 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb2aae7-4e0f-4f41-be74-6f8e2ea36821 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:58.662981 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827a5b4e-6be1-4509-940e-50984a83c777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:58.663873 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a5b4e-6be1-4509-940e-50984a83c777 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:58.664298 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a5b4e-6be1-4509-940e-50984a83c777 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:58.664783 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a5b4e-6be1-4509-940e-50984a83c777 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:58.674443 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92295e4-8d97-4e74-b0aa-f2313306bc9a None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:58.722829 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b31d30-844f-4671-ae74-42ddd6c7b4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:58.723576 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b31d30-844f-4671-ae74-42ddd6c7b4dd None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:58.723791 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b31d30-844f-4671-ae74-42ddd6c7b4dd None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:58.724070 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b31d30-844f-4671-ae74-42ddd6c7b4dd None admin] PATH_INFO: `/v3/projects/4c7185cb4791495281d3a16293793beb` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:58.727976 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b31d30-844f-4671-ae74-42ddd6c7b4dd None admin] RBAC: Authorizing `identity:get_project(project_id=4c7185cb4791495281d3a16293793beb)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:58.728118 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b31d30-844f-4671-ae74-42ddd6c7b4dd None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:58.730196 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 172/347] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:23:58 2022] GET /identity/v3/projects/4c7185cb4791495281d3a16293793beb => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:58.732155 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4f3d1-942a-405e-a753-aad7aaad8994 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:58.732546 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4f3d1-942a-405e-a753-aad7aaad8994 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:58.732741 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4f3d1-942a-405e-a753-aad7aaad8994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:58.756136 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-d4a4f3d1-942a-405e-a753-aad7aaad8994 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:58.757391 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-d4a4f3d1-942a-405e-a753-aad7aaad8994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:58.761776 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827a5b4e-6be1-4509-940e-50984a83c777 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:58.763360 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827a5b4e-6be1-4509-940e-50984a83c777 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:58.816141 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:58.817565 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:58.855093 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 173/348] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:58.857254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4f3d1-942a-405e-a753-aad7aaad8994 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:58.911808 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 176/349] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:23:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 05:23:58.914119 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f780f37-65db-443a-a91d-573c9c7352df None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:58.914602 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f780f37-65db-443a-a91d-573c9c7352df None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:58.918975 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f780f37-65db-443a-a91d-573c9c7352df None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:58.918975 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 177/350] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:58.918975 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f780f37-65db-443a-a91d-573c9c7352df None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:58.996586 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4f3d1-942a-405e-a753-aad7aaad8994 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:59.038781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87115776-868c-4a8d-a937-48b332568ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:59.039589 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87115776-868c-4a8d-a937-48b332568ed8 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:59.039968 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87115776-868c-4a8d-a937-48b332568ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:59.040340 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87115776-868c-4a8d-a937-48b332568ed8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:59.053561 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87115776-868c-4a8d-a937-48b332568ed8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:23:59.059468 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-87115776-868c-4a8d-a937-48b332568ed8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:23:59.059468 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:23:59.059468 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:23:59.059468 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:23:59.059468 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:23:59.059468 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:23:59.059468 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:23:59.059468 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:23:59.059468 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:23:59.059468 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:23:59.059468 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:23:59.059468 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:23:59.059468 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:23:59.059468 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:23:59.059468 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:23:59.059468 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:23:59.060413 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:23:59.060413 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:23:59.060413 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:23:59.060413 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:23:59.060413 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:23:59.060413 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:23:59.060413 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:23:59.060413 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:23:59.060413 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:23:59.060413 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:23:59.060413 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:23:59.060413 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:23:59.060413 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:23:59.060413 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:23:59.060413 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:23:59.060413 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:23:59.061234 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:23:59.061234 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:23:59.061234 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:23:59.061234 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:23:59.061234 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:23:59.061234 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:23:59.061234 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:23:59.061234 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:23:59.061234 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:23:59.061234 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:23:59.061234 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:23:59.061234 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:23:59.061234 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 174/351] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:23:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:23:59.061234 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87115776-868c-4a8d-a937-48b332568ed8 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:59.069395 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f780f37-65db-443a-a91d-573c9c7352df None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:59.105238 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:59.115154 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52693bf9-3265-4891-bab2-e33b3784acd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:59.122991 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52693bf9-3265-4891-bab2-e33b3784acd0 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:59.122991 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52693bf9-3265-4891-bab2-e33b3784acd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:59.122991 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52693bf9-3265-4891-bab2-e33b3784acd0 None admin] PATH_INFO: `/v3/groups/abcb375861e44c97ba4d47a68e63ee18` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:59.125332 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52693bf9-3265-4891-bab2-e33b3784acd0 None admin] RBAC: Authorizing `identity:get_group(group_id=abcb375861e44c97ba4d47a68e63ee18)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:59.127511 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52693bf9-3265-4891-bab2-e33b3784acd0 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:59.129951 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 178/352] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Aug 24 05:23:58 2022] GET /identity/v3/groups/abcb375861e44c97ba4d47a68e63ee18 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:59.131919 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb3fb9-2203-4335-a80a-cd14dd00d631 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:59.132206 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb3fb9-2203-4335-a80a-cd14dd00d631 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:59.132515 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb3fb9-2203-4335-a80a-cd14dd00d631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:59.143150 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:59.155391 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-7dcb3fb9-2203-4335-a80a-cd14dd00d631 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:59.156878 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-7dcb3fb9-2203-4335-a80a-cd14dd00d631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:59.188685 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87115776-868c-4a8d-a937-48b332568ed8 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:59.231899 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f47e6-a7ca-4d30-9009-773e61990cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:59.232628 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f47e6-a7ca-4d30-9009-773e61990cac None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:59.232918 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f47e6-a7ca-4d30-9009-773e61990cac None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:59.234195 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f47e6-a7ca-4d30-9009-773e61990cac None admin] PATH_INFO: `/v3/projects/4c7185cb4791495281d3a16293793beb` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:59.235843 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15f47e6-a7ca-4d30-9009-773e61990cac None admin] RBAC: Authorizing `identity:get_project(project_id=4c7185cb4791495281d3a16293793beb)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:59.237347 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15f47e6-a7ca-4d30-9009-773e61990cac None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:59.239690 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 175/353] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:23:59 2022] GET /identity/v3/projects/4c7185cb4791495281d3a16293793beb => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:59.240331 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa8dd8-c25e-41e0-861f-c30ffbf6c635 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:59.240644 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa8dd8-c25e-41e0-861f-c30ffbf6c635 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:59.241060 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa8dd8-c25e-41e0-861f-c30ffbf6c635 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:59.242425 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 176/354] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:23:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:23:59.243092 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa8dd8-c25e-41e0-861f-c30ffbf6c635 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:59.260959 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 179/355] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:59.262191 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb3fb9-2203-4335-a80a-cd14dd00d631 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:59.393943 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa8dd8-c25e-41e0-861f-c30ffbf6c635 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:59.394687 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb3fb9-2203-4335-a80a-cd14dd00d631 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:59.395750 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:59.439722 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:59.446971 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b05384b-f5ff-4bce-b3a3-134e7c0f27b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:59.452159 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05384b-f5ff-4bce-b3a3-134e7c0f27b3 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:59.452159 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05384b-f5ff-4bce-b3a3-134e7c0f27b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:59.452159 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05384b-f5ff-4bce-b3a3-134e7c0f27b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:59.452159 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b05384b-f5ff-4bce-b3a3-134e7c0f27b3 None admin] RBAC: Adding query filter params (group.id=abcb375861e44c97ba4d47a68e63ee18, role.id=87862cd6eaeb4d5fae089a801aac4f87, scope.project.id=4c7185cb4791495281d3a16293793beb) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:59.452159 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b05384b-f5ff-4bce-b3a3-134e7c0f27b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:59.452159 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b05384b-f5ff-4bce-b3a3-134e7c0f27b3 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:59.453851 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2167a192-019c-4883-bc7c-ab6750b62e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:59.454467 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2167a192-019c-4883-bc7c-ab6750b62e43 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:59.454747 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2167a192-019c-4883-bc7c-ab6750b62e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:59.455018 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2167a192-019c-4883-bc7c-ab6750b62e43 None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:59.455695 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2167a192-019c-4883-bc7c-ab6750b62e43 None admin] RBAC: Adding query filter params (name=member) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:23:59.455998 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2167a192-019c-4883-bc7c-ab6750b62e43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:59.457384 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2167a192-019c-4883-bc7c-ab6750b62e43 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:59.462577 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 180/356] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Aug 24 05:23:59 2022] GET /identity/v3/role_assignments?group.id=abcb375861e44c97ba4d47a68e63ee18&scope.project.id=4c7185cb4791495281d3a16293793beb&role.id=87862cd6eaeb4d5fae089a801aac4f87 => generated 260 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:59.463979 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b05384b-f5ff-4bce-b3a3-134e7c0f27b3 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:59.468363 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 177/357] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Aug 24 05:23:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:59.469580 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2167a192-019c-4883-bc7c-ab6750b62e43 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:59.610026 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b05384b-f5ff-4bce-b3a3-134e7c0f27b3 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:59.634072 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2167a192-019c-4883-bc7c-ab6750b62e43 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:59.654260 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbf7ea8-6277-44e8-94fc-c63d6a0f0381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:59.655044 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf7ea8-6277-44e8-94fc-c63d6a0f0381 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:59.655523 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf7ea8-6277-44e8-94fc-c63d6a0f0381 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:59.656892 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf7ea8-6277-44e8-94fc-c63d6a0f0381 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:59.676468 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbf7ea8-6277-44e8-94fc-c63d6a0f0381 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:23:59.683271 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-1cbf7ea8-6277-44e8-94fc-c63d6a0f0381 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:23:59.683271 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:23:59.683271 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:23:59.683271 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:23:59.683271 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:23:59.683271 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:23:59.683271 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:23:59.683271 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:23:59.683271 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:23:59.683271 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:23:59.683271 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:23:59.683271 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:23:59.683271 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:23:59.683271 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:23:59.683271 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:23:59.683271 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:23:59.684170 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:23:59.684170 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:23:59.684170 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:23:59.684170 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:23:59.684170 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:23:59.684170 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:23:59.684170 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:23:59.684170 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:23:59.684170 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:23:59.684170 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:23:59.684170 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:23:59.684170 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:23:59.684170 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:23:59.684170 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:23:59.684170 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:23:59.684170 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:23:59.685267 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:23:59.685267 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:23:59.685267 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:23:59.685267 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:23:59.685267 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:23:59.685267 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:23:59.685267 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:23:59.685267 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:23:59.685267 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:23:59.685267 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:23:59.685267 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:23:59.685267 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:23:59.685851 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee0d5a-e5b2-44fb-b338-df0bf5227430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:59.685851 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee0d5a-e5b2-44fb-b338-df0bf5227430 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:59.686005 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 181/358] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:23:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:23:59.686070 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee0d5a-e5b2-44fb-b338-df0bf5227430 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:59.686070 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee0d5a-e5b2-44fb-b338-df0bf5227430 None admin] PATH_INFO: `/v3/groups/abcb375861e44c97ba4d47a68e63ee18` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:59.686193 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc059d7-c163-485c-815b-83f34b82609f None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:59.686193 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc059d7-c163-485c-815b-83f34b82609f None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:59.686717 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc059d7-c163-485c-815b-83f34b82609f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:59.688741 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ee0d5a-e5b2-44fb-b338-df0bf5227430 None admin] RBAC: Authorizing `identity:get_group(group_id=abcb375861e44c97ba4d47a68e63ee18)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:59.689818 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ee0d5a-e5b2-44fb-b338-df0bf5227430 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:23:59.696632 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 178/359] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Aug 24 05:23:59 2022] GET /identity/v3/groups/abcb375861e44c97ba4d47a68e63ee18 => generated 226 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:23:59.697999 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee0d5a-e5b2-44fb-b338-df0bf5227430 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:59.713975 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-ddc059d7-c163-485c-815b-83f34b82609f None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:23:59.715180 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-ddc059d7-c163-485c-815b-83f34b82609f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:23:59.743302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:59.790762 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:23:59.811888 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 182/360] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:23:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:23:59.813193 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc059d7-c163-485c-815b-83f34b82609f None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:59.851379 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee0d5a-e5b2-44fb-b338-df0bf5227430 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:59.895232 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ea694-50ee-4154-a3ae-c3e691a7c2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:59.895745 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ea694-50ee-4154-a3ae-c3e691a7c2a8 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:59.896036 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ea694-50ee-4154-a3ae-c3e691a7c2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:59.896311 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ea694-50ee-4154-a3ae-c3e691a7c2a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:59.919662 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc059d7-c163-485c-815b-83f34b82609f None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:23:59.962606 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166d0e91-38ad-4ace-b48c-8771b61fe98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:23:59.963255 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d0e91-38ad-4ace-b48c-8771b61fe98d None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:23:59.963532 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d0e91-38ad-4ace-b48c-8771b61fe98d None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:23:59.963793 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d0e91-38ad-4ace-b48c-8771b61fe98d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:23:59.977104 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166d0e91-38ad-4ace-b48c-8771b61fe98d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:23:59.981893 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-166d0e91-38ad-4ace-b48c-8771b61fe98d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:23:59.981893 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:23:59.981893 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:23:59.981893 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:23:59.981893 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:23:59.981893 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:23:59.981893 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:23:59.981893 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:23:59.981893 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:23:59.981893 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:23:59.981893 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:23:59.981893 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:23:59.981893 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:23:59.981893 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:23:59.981893 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:23:59.981893 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:23:59.982791 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:23:59.982791 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:23:59.982791 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:23:59.982791 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:23:59.982791 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:23:59.982791 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:23:59.982791 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:23:59.982791 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:23:59.982791 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:23:59.982791 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:23:59.982791 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:23:59.982791 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:23:59.982791 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:23:59.982791 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:23:59.982791 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:23:59.982791 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:23:59.983580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:23:59.983580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:23:59.983580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:23:59.983580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:23:59.983580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:23:59.983580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:23:59.983580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:23:59.983580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:23:59.983580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:23:59.983580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:23:59.983580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:23:59.983580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:23:59.983580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 183/361] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Aug 24 05:23:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:23:59.984518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166d0e91-38ad-4ace-b48c-8771b61fe98d None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:23:59.989492 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22ea694-50ee-4154-a3ae-c3e691a7c2a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:23:59.991303 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22ea694-50ee-4154-a3ae-c3e691a7c2a8 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:00.027664 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:00.035723 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:00.092215 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166d0e91-38ad-4ace-b48c-8771b61fe98d None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:00.127907 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 179/362] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:23:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:24:00.128809 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ea694-50ee-4154-a3ae-c3e691a7c2a8 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:00.134888 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f65ddf3-7767-4551-9f1a-94ede0da218b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:00.135517 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f65ddf3-7767-4551-9f1a-94ede0da218b None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:00.135800 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f65ddf3-7767-4551-9f1a-94ede0da218b None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:00.136077 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f65ddf3-7767-4551-9f1a-94ede0da218b None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:00.136804 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f65ddf3-7767-4551-9f1a-94ede0da218b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:00.137112 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f65ddf3-7767-4551-9f1a-94ede0da218b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:00.138521 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f65ddf3-7767-4551-9f1a-94ede0da218b None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:00.151813 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 184/363] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Aug 24 05:23:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:00.152730 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f65ddf3-7767-4551-9f1a-94ede0da218b None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:00.272092 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ea694-50ee-4154-a3ae-c3e691a7c2a8 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:00.283599 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f65ddf3-7767-4551-9f1a-94ede0da218b None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:00.295583 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:00.308968 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:00.315972 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bff0223-a041-4f90-8912-2dc84a0240be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:00.317750 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bff0223-a041-4f90-8912-2dc84a0240be None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:00.318035 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bff0223-a041-4f90-8912-2dc84a0240be None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:00.318305 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bff0223-a041-4f90-8912-2dc84a0240be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:00.319116 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bff0223-a041-4f90-8912-2dc84a0240be None admin] RBAC: Adding query filter params (group.id=abcb375861e44c97ba4d47a68e63ee18, role.id=0cdb52ffa00449399b27dbfcd383f488, scope.project.id=4c7185cb4791495281d3a16293793beb) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:00.319563 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bff0223-a041-4f90-8912-2dc84a0240be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:00.321024 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bff0223-a041-4f90-8912-2dc84a0240be None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:00.322308 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f68806-2632-4103-b0da-99bbf09d42fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:00.323131 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f68806-2632-4103-b0da-99bbf09d42fe None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:00.323585 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f68806-2632-4103-b0da-99bbf09d42fe None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:00.324055 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f68806-2632-4103-b0da-99bbf09d42fe None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:00.324897 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f68806-2632-4103-b0da-99bbf09d42fe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:00.325215 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f68806-2632-4103-b0da-99bbf09d42fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:00.327543 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f68806-2632-4103-b0da-99bbf09d42fe None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:00.328337 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 180/364] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Aug 24 05:24:00 2022] GET /identity/v3/role_assignments?group.id=abcb375861e44c97ba4d47a68e63ee18&scope.project.id=4c7185cb4791495281d3a16293793beb&role.id=0cdb52ffa00449399b27dbfcd383f488 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:00.329240 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bff0223-a041-4f90-8912-2dc84a0240be None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:00.348269 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 185/365] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:24:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:00.349915 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f68806-2632-4103-b0da-99bbf09d42fe None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:00.480596 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f68806-2632-4103-b0da-99bbf09d42fe None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:00.486870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bff0223-a041-4f90-8912-2dc84a0240be None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:00.525794 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724650b6-ddfb-49b1-8f36-e9ac6672a31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:00.526492 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724650b6-ddfb-49b1-8f36-e9ac6672a31e None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:00.526912 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724650b6-ddfb-49b1-8f36-e9ac6672a31e None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:00.527189 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724650b6-ddfb-49b1-8f36-e9ac6672a31e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:00.536329 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf876ef-87c5-4a5a-941e-a87bb0b4e0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:00.537546 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf876ef-87c5-4a5a-941e-a87bb0b4e0ec None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:00.537992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf876ef-87c5-4a5a-941e-a87bb0b4e0ec None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:00.538419 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf876ef-87c5-4a5a-941e-a87bb0b4e0ec None admin] PATH_INFO: `/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:00.539861 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf876ef-87c5-4a5a-941e-a87bb0b4e0ec None admin] RBAC: Authorizing `identity:get_project(project_id=a7f9d22fd79e4b1fa16a2f60fc6faec7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:00.541537 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf876ef-87c5-4a5a-941e-a87bb0b4e0ec None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:00.545234 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 181/366] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:24:00 2022] GET /identity/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:00.545234 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf876ef-87c5-4a5a-941e-a87bb0b4e0ec None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:00.546818 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724650b6-ddfb-49b1-8f36-e9ac6672a31e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:24:00.551907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-724650b6-ddfb-49b1-8f36-e9ac6672a31e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:24:00.551907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:00.551907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:00.551907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:00.551907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:00.551907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:00.551907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:00.551907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:00.551907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:00.551907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:00.551907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:00.551907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:00.551907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:24:00.551907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:24:00.551907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:24:00.551907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:24:00.552857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:24:00.552857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:24:00.552857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:24:00.552857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:24:00.552857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:00.552857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:00.552857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:00.552857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:00.552857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:00.552857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:00.552857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:00.552857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:00.552857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:00.552857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:00.552857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:00.552857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:00.553647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:00.553647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:00.553647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:00.553647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:00.553647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:24:00.553647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:24:00.553647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:24:00.553647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:24:00.553647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:24:00.553647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:24:00.553647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:24:00.553647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:00.553647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 186/367] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Aug 24 05:24:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:00.554302 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724650b6-ddfb-49b1-8f36-e9ac6672a31e None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:00.591788 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:00.656623 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:00.688261 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724650b6-ddfb-49b1-8f36-e9ac6672a31e None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:00.701133 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf876ef-87c5-4a5a-941e-a87bb0b4e0ec None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:00.729367 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b31567-f911-4378-b47f-ce42ff4c32bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:00.730003 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b31567-f911-4378-b47f-ce42ff4c32bc None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:00.730281 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b31567-f911-4378-b47f-ce42ff4c32bc None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:00.730554 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b31567-f911-4378-b47f-ce42ff4c32bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:00.742223 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2217bb-e97b-4bbd-b2c0-afb694404767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:00.742803 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2217bb-e97b-4bbd-b2c0-afb694404767 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:00.743087 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2217bb-e97b-4bbd-b2c0-afb694404767 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:00.743450 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2217bb-e97b-4bbd-b2c0-afb694404767 None admin] PATH_INFO: `/v3/users/adbcbc85ba344aeb9e669954f6c63f3c` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:00.744745 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2217bb-e97b-4bbd-b2c0-afb694404767 None admin] RBAC: Authorizing `identity:get_user(user_id=adbcbc85ba344aeb9e669954f6c63f3c)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:00.746319 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2217bb-e97b-4bbd-b2c0-afb694404767 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:00.748430 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 182/368] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:24:00 2022] GET /identity/v3/users/adbcbc85ba344aeb9e669954f6c63f3c => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:00.749348 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2217bb-e97b-4bbd-b2c0-afb694404767 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:00.806634 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b31567-f911-4378-b47f-ce42ff4c32bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:00.808217 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b31567-f911-4378-b47f-ce42ff4c32bc None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:00.840083 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:00.864226 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2217bb-e97b-4bbd-b2c0-afb694404767 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:00.897110 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:00.902546 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a06725-6488-40a3-a2d1-4bf3347b3bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:00.903340 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a06725-6488-40a3-a2d1-4bf3347b3bba None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:00.903697 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a06725-6488-40a3-a2d1-4bf3347b3bba None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:00.904048 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a06725-6488-40a3-a2d1-4bf3347b3bba None admin] PATH_INFO: `/v3/groups/abcb375861e44c97ba4d47a68e63ee18` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:00.905257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a06725-6488-40a3-a2d1-4bf3347b3bba None admin] RBAC: Authorizing `identity:get_group(group_id=abcb375861e44c97ba4d47a68e63ee18)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:00.907657 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a06725-6488-40a3-a2d1-4bf3347b3bba None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:00.909469 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 183/369] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Aug 24 05:24:00 2022] GET /identity/v3/groups/abcb375861e44c97ba4d47a68e63ee18 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:00.915031 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a06725-6488-40a3-a2d1-4bf3347b3bba None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:00.920996 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 187/370] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:24:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:24:00.922413 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b31567-f911-4378-b47f-ce42ff4c32bc None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:01.027864 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a06725-6488-40a3-a2d1-4bf3347b3bba None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:01.039313 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b31567-f911-4378-b47f-ce42ff4c32bc None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:01.069499 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0454780-f95e-4775-8957-38d0f8786921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:01.070114 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0454780-f95e-4775-8957-38d0f8786921 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:01.070399 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0454780-f95e-4775-8957-38d0f8786921 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:01.070676 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0454780-f95e-4775-8957-38d0f8786921 None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:01.071393 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:01.071867 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0454780-f95e-4775-8957-38d0f8786921 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:01.073034 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0454780-f95e-4775-8957-38d0f8786921 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:01.074602 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0454780-f95e-4775-8957-38d0f8786921 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:01.078712 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b96d16-aeb4-4ba9-af38-3c0f5175126b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:01.079385 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b96d16-aeb4-4ba9-af38-3c0f5175126b None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:01.079643 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b96d16-aeb4-4ba9-af38-3c0f5175126b None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:01.083840 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b96d16-aeb4-4ba9-af38-3c0f5175126b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:01.083840 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b96d16-aeb4-4ba9-af38-3c0f5175126b None admin] RBAC: Adding query filter params (role.id=49b52d60e4434f05870a37c49b2e6e6b, user.id=adbcbc85ba344aeb9e669954f6c63f3c, scope.system=all) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:01.083840 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b96d16-aeb4-4ba9-af38-3c0f5175126b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:01.083840 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b96d16-aeb4-4ba9-af38-3c0f5175126b None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:01.086057 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 184/371] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:24:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:01.087187 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0454780-f95e-4775-8957-38d0f8786921 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:01.097252 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 188/372] 10.222.0.15 () {56 vars in 1308 bytes} [Wed Aug 24 05:24:00 2022] GET /identity/v3/role_assignments?user.id=adbcbc85ba344aeb9e669954f6c63f3c&scope.system=all&role.id=49b52d60e4434f05870a37c49b2e6e6b => generated 510 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:01.098095 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b96d16-aeb4-4ba9-af38-3c0f5175126b None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:01.161165 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:01.212286 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0454780-f95e-4775-8957-38d0f8786921 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:01.227755 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b96d16-aeb4-4ba9-af38-3c0f5175126b None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:01.258330 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091e82c0-2fc8-4dc1-9557-6a7822ce0af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:01.259158 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e82c0-2fc8-4dc1-9557-6a7822ce0af2 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:01.259500 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e82c0-2fc8-4dc1-9557-6a7822ce0af2 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:01.259790 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e82c0-2fc8-4dc1-9557-6a7822ce0af2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:01.260762 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091e82c0-2fc8-4dc1-9557-6a7822ce0af2 None admin] RBAC: Adding query filter params (group.id=abcb375861e44c97ba4d47a68e63ee18, role.id=87862cd6eaeb4d5fae089a801aac4f87, scope.project.id=a7f9d22fd79e4b1fa16a2f60fc6faec7) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:01.261083 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091e82c0-2fc8-4dc1-9557-6a7822ce0af2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:01.262675 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091e82c0-2fc8-4dc1-9557-6a7822ce0af2 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:01.270548 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5028895-a980-4778-91a7-45aa68f7ce20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:01.270874 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 185/373] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Aug 24 05:24:01 2022] GET /identity/v3/role_assignments?group.id=abcb375861e44c97ba4d47a68e63ee18&scope.project.id=a7f9d22fd79e4b1fa16a2f60fc6faec7&role.id=87862cd6eaeb4d5fae089a801aac4f87 => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:01.271391 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5028895-a980-4778-91a7-45aa68f7ce20 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:01.271724 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091e82c0-2fc8-4dc1-9557-6a7822ce0af2 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:01.271841 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5028895-a980-4778-91a7-45aa68f7ce20 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:01.272064 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5028895-a980-4778-91a7-45aa68f7ce20 None admin] PATH_INFO: `/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:01.275505 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5028895-a980-4778-91a7-45aa68f7ce20 None admin] RBAC: Authorizing `identity:get_project(project_id=a7f9d22fd79e4b1fa16a2f60fc6faec7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:01.277297 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5028895-a980-4778-91a7-45aa68f7ce20 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:01.279290 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 189/374] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:24:01 2022] GET /identity/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:01.285520 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5028895-a980-4778-91a7-45aa68f7ce20 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:01.358849 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:01.410234 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5028895-a980-4778-91a7-45aa68f7ce20 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:01.455356 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec4455-fb78-4ee4-b754-bdfaf122c98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:01.456013 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec4455-fb78-4ee4-b754-bdfaf122c98d None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:01.456299 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec4455-fb78-4ee4-b754-bdfaf122c98d None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:01.458213 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec4455-fb78-4ee4-b754-bdfaf122c98d None admin] PATH_INFO: `/v3/groups/abcb375861e44c97ba4d47a68e63ee18` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:01.460381 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eec4455-fb78-4ee4-b754-bdfaf122c98d None admin] RBAC: Authorizing `identity:get_group(group_id=abcb375861e44c97ba4d47a68e63ee18)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:01.461888 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eec4455-fb78-4ee4-b754-bdfaf122c98d None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:01.463728 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 190/375] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Aug 24 05:24:01 2022] GET /identity/v3/groups/abcb375861e44c97ba4d47a68e63ee18 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:24:01.472214 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec4455-fb78-4ee4-b754-bdfaf122c98d None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:01.478822 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091e82c0-2fc8-4dc1-9557-6a7822ce0af2 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:01.534059 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:01.557303 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab3457-d061-4a20-a32d-b846a6892b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:01.557985 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab3457-d061-4a20-a32d-b846a6892b1f None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:01.558324 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab3457-d061-4a20-a32d-b846a6892b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:01.558606 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab3457-d061-4a20-a32d-b846a6892b1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:01.634184 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec4455-fb78-4ee4-b754-bdfaf122c98d None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:01.665850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:01.679619 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a07b3-d29f-44b8-9c36-ab78421fd23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:01.680424 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a07b3-d29f-44b8-9c36-ab78421fd23b None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:01.680823 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a07b3-d29f-44b8-9c36-ab78421fd23b None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:01.681100 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a07b3-d29f-44b8-9c36-ab78421fd23b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:01.681835 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81a07b3-d29f-44b8-9c36-ab78421fd23b None admin] RBAC: Adding query filter params (group.id=abcb375861e44c97ba4d47a68e63ee18, role.id=0cdb52ffa00449399b27dbfcd383f488, scope.project.id=a7f9d22fd79e4b1fa16a2f60fc6faec7) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:01.682145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81a07b3-d29f-44b8-9c36-ab78421fd23b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:01.683702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81a07b3-d29f-44b8-9c36-ab78421fd23b None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:01.690101 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 191/376] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Aug 24 05:24:01 2022] GET /identity/v3/role_assignments?group.id=abcb375861e44c97ba4d47a68e63ee18&scope.project.id=a7f9d22fd79e4b1fa16a2f60fc6faec7&role.id=0cdb52ffa00449399b27dbfcd383f488 => generated 260 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:01.721461 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ab3457-d061-4a20-a32d-b846a6892b1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:01.722989 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ab3457-d061-4a20-a32d-b846a6892b1f None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:01.875589 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 186/377] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:24:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:24:01.881688 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a07b3-d29f-44b8-9c36-ab78421fd23b None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:01.964645 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d01504-bd85-4578-a5e3-49b758795927 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:01.964959 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d01504-bd85-4578-a5e3-49b758795927 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:01.965205 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d01504-bd85-4578-a5e3-49b758795927 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:01.966788 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 187/378] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:01.972939 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af03ac-76f8-485b-b3f1-d226bdefddba None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:01.973258 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af03ac-76f8-485b-b3f1-d226bdefddba None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:01.973544 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af03ac-76f8-485b-b3f1-d226bdefddba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:02.003904 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-14af03ac-76f8-485b-b3f1-d226bdefddba None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:02.005645 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-14af03ac-76f8-485b-b3f1-d226bdefddba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:02.035075 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:02.050598 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a07b3-d29f-44b8-9c36-ab78421fd23b None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:02.098582 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 188/379] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:24:02.118144 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7ec554-dbda-4e86-bdbc-579ca5b2dd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:02.118796 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7ec554-dbda-4e86-bdbc-579ca5b2dd19 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:02.119084 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7ec554-dbda-4e86-bdbc-579ca5b2dd19 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:02.119365 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7ec554-dbda-4e86-bdbc-579ca5b2dd19 None admin] PATH_INFO: `/v3/projects/617e47339cb94e20967707fe62db290d` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:02.120552 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7ec554-dbda-4e86-bdbc-579ca5b2dd19 None admin] RBAC: Authorizing `identity:get_project(project_id=617e47339cb94e20967707fe62db290d)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:02.122001 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7ec554-dbda-4e86-bdbc-579ca5b2dd19 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:02.123975 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 192/380] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:24:01 2022] GET /identity/v3/projects/617e47339cb94e20967707fe62db290d => generated 344 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:02.130654 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af03ac-76f8-485b-b3f1-d226bdefddba None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:02.275991 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af03ac-76f8-485b-b3f1-d226bdefddba None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:02.327436 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ea2cb-b5ba-4909-955a-d45471c1bdeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:02.328085 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ea2cb-b5ba-4909-955a-d45471c1bdeb None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:02.328371 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ea2cb-b5ba-4909-955a-d45471c1bdeb None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:02.328672 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ea2cb-b5ba-4909-955a-d45471c1bdeb None admin] PATH_INFO: `/v3/groups/0371c18296e84248adae66efa0f5a092` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:02.348036 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73ea2cb-b5ba-4909-955a-d45471c1bdeb None admin] RBAC: Authorizing `identity:get_group(group_id=0371c18296e84248adae66efa0f5a092)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:02.350336 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73ea2cb-b5ba-4909-955a-d45471c1bdeb None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:02.353340 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 189/381] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Aug 24 05:24:02 2022] GET /identity/v3/groups/0371c18296e84248adae66efa0f5a092 => generated 229 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:24:02.360215 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7ec554-dbda-4e86-bdbc-579ca5b2dd19 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:02.423526 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ea2cb-b5ba-4909-955a-d45471c1bdeb None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:02.436840 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:02.471420 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:02.511201 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7ec554-dbda-4e86-bdbc-579ca5b2dd19 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:02.562608 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03853192-d939-4789-8499-13e0721a7c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:02.563293 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03853192-d939-4789-8499-13e0721a7c56 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:02.563533 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03853192-d939-4789-8499-13e0721a7c56 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:02.563711 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03853192-d939-4789-8499-13e0721a7c56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:02.564637 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03853192-d939-4789-8499-13e0721a7c56 None admin] RBAC: Adding query filter params (group.id=0371c18296e84248adae66efa0f5a092, role.id=e013ad4d436f4cb0a9bf7debb7d13dc7, scope.project.id=617e47339cb94e20967707fe62db290d) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:02.565079 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03853192-d939-4789-8499-13e0721a7c56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:02.566903 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03853192-d939-4789-8499-13e0721a7c56 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:02.577395 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 193/382] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Aug 24 05:24:02 2022] GET /identity/v3/role_assignments?group.id=0371c18296e84248adae66efa0f5a092&scope.project.id=617e47339cb94e20967707fe62db290d&role.id=e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 260 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:02.609894 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ea2cb-b5ba-4909-955a-d45471c1bdeb None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:02.649059 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f82701-7039-4e0e-9e6e-ec71c96b680a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:02.649771 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f82701-7039-4e0e-9e6e-ec71c96b680a None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:02.650049 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f82701-7039-4e0e-9e6e-ec71c96b680a None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:02.650322 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f82701-7039-4e0e-9e6e-ec71c96b680a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:02.662671 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f82701-7039-4e0e-9e6e-ec71c96b680a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:24:02.667915 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-a1f82701-7039-4e0e-9e6e-ec71c96b680a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:24:02.667915 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:02.667915 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:02.667915 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:02.667915 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:02.667915 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:02.667915 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:02.667915 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:02.667915 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:02.667915 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:02.667915 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:02.667915 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:02.667915 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:24:02.667915 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:24:02.667915 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:24:02.667915 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:24:02.668978 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:24:02.668978 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:24:02.668978 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:24:02.668978 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:24:02.668978 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:02.668978 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:02.668978 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:02.668978 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:02.668978 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:02.668978 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:02.668978 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:02.668978 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:02.668978 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:02.668978 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:02.668978 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:02.668978 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:02.669768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:02.669768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:02.669768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:02.669768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:02.669768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:24:02.669768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:24:02.669768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:24:02.669768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:24:02.669768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:24:02.669768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:24:02.669768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:24:02.669768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:02.669768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 190/383] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:24:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:24:02.675838 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03853192-d939-4789-8499-13e0721a7c56 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:02.835555 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03853192-d939-4789-8499-13e0721a7c56 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:02.857879 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:02.876647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22465358-90e8-4feb-a420-f2b6e3da2dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:02.877272 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22465358-90e8-4feb-a420-f2b6e3da2dd1 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:02.877558 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22465358-90e8-4feb-a420-f2b6e3da2dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:02.877834 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22465358-90e8-4feb-a420-f2b6e3da2dd1 None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:02.878507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22465358-90e8-4feb-a420-f2b6e3da2dd1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:02.878814 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22465358-90e8-4feb-a420-f2b6e3da2dd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:02.880274 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22465358-90e8-4feb-a420-f2b6e3da2dd1 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:02.890061 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 194/384] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Aug 24 05:24:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:02.895589 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f82701-7039-4e0e-9e6e-ec71c96b680a None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:02.925717 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfda17-9fa9-430c-80a2-949de6fc846d None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:02.926065 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfda17-9fa9-430c-80a2-949de6fc846d None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:02.926275 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfda17-9fa9-430c-80a2-949de6fc846d None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:02.927844 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 195/385] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:02.933611 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51b086-5edb-437e-a385-239e0b7cea86 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:02.934037 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51b086-5edb-437e-a385-239e0b7cea86 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:02.934319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51b086-5edb-437e-a385-239e0b7cea86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:02.956452 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-3c51b086-5edb-437e-a385-239e0b7cea86 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:02.958364 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-3c51b086-5edb-437e-a385-239e0b7cea86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:03.033274 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:03.044840 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 196/386] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:24:03.076188 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f82701-7039-4e0e-9e6e-ec71c96b680a None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:03.121200 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c379294-ad62-4312-932d-a36382cbfa6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:03.121839 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c379294-ad62-4312-932d-a36382cbfa6b None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:03.122114 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c379294-ad62-4312-932d-a36382cbfa6b None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:03.122384 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c379294-ad62-4312-932d-a36382cbfa6b None admin] PATH_INFO: `/v3/groups/abcb375861e44c97ba4d47a68e63ee18` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:03.123499 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c379294-ad62-4312-932d-a36382cbfa6b None admin] RBAC: Authorizing `identity:get_group(group_id=abcb375861e44c97ba4d47a68e63ee18)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:03.125062 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c379294-ad62-4312-932d-a36382cbfa6b None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:03.127126 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 191/387] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Aug 24 05:24:02 2022] GET /identity/v3/groups/abcb375861e44c97ba4d47a68e63ee18 => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:24:03.132614 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c51b086-5edb-437e-a385-239e0b7cea86 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:03.284479 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c51b086-5edb-437e-a385-239e0b7cea86 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:03.302413 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:03.336190 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f4f9a2-9b66-40f3-87e6-6a20adcf1c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:03.336876 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4f9a2-9b66-40f3-87e6-6a20adcf1c96 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:03.337160 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4f9a2-9b66-40f3-87e6-6a20adcf1c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:03.337441 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4f9a2-9b66-40f3-87e6-6a20adcf1c96 None admin] PATH_INFO: `/v3/projects/4c7185cb4791495281d3a16293793beb` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:03.341714 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f4f9a2-9b66-40f3-87e6-6a20adcf1c96 None admin] RBAC: Authorizing `identity:get_project(project_id=4c7185cb4791495281d3a16293793beb)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:03.343245 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f4f9a2-9b66-40f3-87e6-6a20adcf1c96 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:03.345876 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 197/388] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:24:03 2022] GET /identity/v3/projects/4c7185cb4791495281d3a16293793beb => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:03.351321 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f4f9a2-9b66-40f3-87e6-6a20adcf1c96 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:03.351561 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c379294-ad62-4312-932d-a36382cbfa6b None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:03.481302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c379294-ad62-4312-932d-a36382cbfa6b None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:03.548389 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd9f2ee-d072-4a6d-a213-19cde81424df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:03.549052 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9f2ee-d072-4a6d-a213-19cde81424df None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:03.549334 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9f2ee-d072-4a6d-a213-19cde81424df None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:03.549609 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9f2ee-d072-4a6d-a213-19cde81424df None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:03.570820 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f4f9a2-9b66-40f3-87e6-6a20adcf1c96 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:03.573892 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd9f2ee-d072-4a6d-a213-19cde81424df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:24:03.587287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-2fd9f2ee-d072-4a6d-a213-19cde81424df None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:24:03.587287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:03.587287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:03.587287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:03.587287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:03.587287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:03.587287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:03.587287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:03.587287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:03.587287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:03.587287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:03.587287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:03.587287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:24:03.587287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:24:03.587287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:24:03.587287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:24:03.589500 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:24:03.589500 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:24:03.589500 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:24:03.589500 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:24:03.589500 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:03.589500 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:03.589500 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:03.589500 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:03.589500 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:03.589500 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:03.589500 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:03.589500 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:03.589500 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:03.589500 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:03.589500 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:03.589500 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:03.591441 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:03.591441 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:03.591441 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:03.591441 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:03.591441 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:24:03.591441 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:24:03.591441 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:24:03.591441 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:24:03.591441 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:24:03.591441 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:24:03.591441 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:24:03.591441 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:03.594632 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 192/389] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Aug 24 05:24:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:03.600097 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd9f2ee-d072-4a6d-a213-19cde81424df None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:03.617223 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc76986-4e14-4051-9fb5-cfc4e1cd8f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:03.617912 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc76986-4e14-4051-9fb5-cfc4e1cd8f38 None admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:03.618194 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc76986-4e14-4051-9fb5-cfc4e1cd8f38 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:03.618508 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc76986-4e14-4051-9fb5-cfc4e1cd8f38 None admin] PATH_INFO: `/v3/projects/4c7185cb4791495281d3a16293793beb/groups/abcb375861e44c97ba4d47a68e63ee18/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:03.620817 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc76986-4e14-4051-9fb5-cfc4e1cd8f38 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c7185cb4791495281d3a16293793beb, group_id=abcb375861e44c97ba4d47a68e63ee18, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:03.622257 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc76986-4e14-4051-9fb5-cfc4e1cd8f38 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:03.627512 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:03.699699 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 198/390] 10.222.0.15 () {58 vars in 1442 bytes} [Wed Aug 24 05:24:03 2022] PUT /identity/v3/projects/4c7185cb4791495281d3a16293793beb/groups/abcb375861e44c97ba4d47a68e63ee18/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:24:03.740979 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd9f2ee-d072-4a6d-a213-19cde81424df None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:03.782135 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f8495-5193-4af8-aa3d-a6523c7f2487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:03.782908 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f8495-5193-4af8-aa3d-a6523c7f2487 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:03.783195 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f8495-5193-4af8-aa3d-a6523c7f2487 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:03.783470 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f8495-5193-4af8-aa3d-a6523c7f2487 None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:03.784195 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220f8495-5193-4af8-aa3d-a6523c7f2487 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:03.784532 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220f8495-5193-4af8-aa3d-a6523c7f2487 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:03.786086 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220f8495-5193-4af8-aa3d-a6523c7f2487 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:03.795783 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 193/391] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:24:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:24:03.801844 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc76986-4e14-4051-9fb5-cfc4e1cd8f38 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:03.855943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:03.920254 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc76986-4e14-4051-9fb5-cfc4e1cd8f38 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:03.971413 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b49ee0d-c6ec-47b6-9dfb-503842b5c225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:03.972139 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49ee0d-c6ec-47b6-9dfb-503842b5c225 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:03.972443 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49ee0d-c6ec-47b6-9dfb-503842b5c225 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:03.972727 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49ee0d-c6ec-47b6-9dfb-503842b5c225 None admin] PATH_INFO: `/v3/groups/abcb375861e44c97ba4d47a68e63ee18` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:03.973868 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b49ee0d-c6ec-47b6-9dfb-503842b5c225 None admin] RBAC: Authorizing `identity:get_group(group_id=abcb375861e44c97ba4d47a68e63ee18)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:03.975315 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b49ee0d-c6ec-47b6-9dfb-503842b5c225 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:03.977098 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 199/392] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Aug 24 05:24:03 2022] GET /identity/v3/groups/abcb375861e44c97ba4d47a68e63ee18 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:03.982783 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f8495-5193-4af8-aa3d-a6523c7f2487 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:04.077683 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:04.113237 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f8495-5193-4af8-aa3d-a6523c7f2487 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:04.128719 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175c32b-c02c-4dcd-9d13-8a42d8730676 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:04.128719 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175c32b-c02c-4dcd-9d13-8a42d8730676 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:04.128719 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175c32b-c02c-4dcd-9d13-8a42d8730676 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:04.130282 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 200/393] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:24:04.135964 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b9960-ef51-46f1-8bf2-c83012495f8b None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:04.136575 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b9960-ef51-46f1-8bf2-c83012495f8b None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:04.136575 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b9960-ef51-46f1-8bf2-c83012495f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:04.159163 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-ba9b9960-ef51-46f1-8bf2-c83012495f8b None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:04.160517 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-ba9b9960-ef51-46f1-8bf2-c83012495f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:04.163169 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0163150c-738a-4317-aa04-3e25395f2c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:04.164066 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163150c-738a-4317-aa04-3e25395f2c0e None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:04.164355 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163150c-738a-4317-aa04-3e25395f2c0e None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:04.164716 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163150c-738a-4317-aa04-3e25395f2c0e None admin] PATH_INFO: `/v3/projects/4c7185cb4791495281d3a16293793beb` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:04.167544 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0163150c-738a-4317-aa04-3e25395f2c0e None admin] RBAC: Authorizing `identity:get_project(project_id=4c7185cb4791495281d3a16293793beb)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:04.169056 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0163150c-738a-4317-aa04-3e25395f2c0e None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:04.171015 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 194/394] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:24:03 2022] GET /identity/v3/projects/4c7185cb4791495281d3a16293793beb => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:04.177784 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0163150c-738a-4317-aa04-3e25395f2c0e None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:04.246756 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 201/395] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:24:04.359726 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0163150c-738a-4317-aa04-3e25395f2c0e None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:04.399787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ebc0b-0178-429c-b1c1-7cac6dba5c8d None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:04.400123 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9378d3c9-afeb-4088-9456-9340816b8033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:04.400323 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ebc0b-0178-429c-b1c1-7cac6dba5c8d None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:04.400503 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ebc0b-0178-429c-b1c1-7cac6dba5c8d None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:04.400818 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9378d3c9-afeb-4088-9456-9340816b8033 None admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:04.401157 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9378d3c9-afeb-4088-9456-9340816b8033 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:04.401434 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9378d3c9-afeb-4088-9456-9340816b8033 None admin] PATH_INFO: `/v3/projects/4c7185cb4791495281d3a16293793beb/groups/abcb375861e44c97ba4d47a68e63ee18/roles/0cdb52ffa00449399b27dbfcd383f488` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:04.402310 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 202/396] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:24:04.406930 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9378d3c9-afeb-4088-9456-9340816b8033 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c7185cb4791495281d3a16293793beb, group_id=abcb375861e44c97ba4d47a68e63ee18, role_id=0cdb52ffa00449399b27dbfcd383f488)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:04.408384 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ac77f-9a51-4911-b371-a1911116e42a None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:04.408516 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9378d3c9-afeb-4088-9456-9340816b8033 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:04.408771 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ac77f-9a51-4911-b371-a1911116e42a None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:04.409116 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ac77f-9a51-4911-b371-a1911116e42a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:04.429333 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-cc3ac77f-9a51-4911-b371-a1911116e42a None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:04.430699 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-cc3ac77f-9a51-4911-b371-a1911116e42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:04.466684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:04.483374 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:04.497319 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 195/397] 10.222.0.15 () {58 vars in 1442 bytes} [Wed Aug 24 05:24:04 2022] PUT /identity/v3/projects/4c7185cb4791495281d3a16293793beb/groups/abcb375861e44c97ba4d47a68e63ee18/roles/0cdb52ffa00449399b27dbfcd383f488 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:24:04.528151 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 203/398] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:24:04.635467 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9378d3c9-afeb-4088-9456-9340816b8033 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:04.758557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9378d3c9-afeb-4088-9456-9340816b8033 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:04.800113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7715dd16-0205-4b1b-b8fc-26d0a7709aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:04.800782 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715dd16-0205-4b1b-b8fc-26d0a7709aae None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:04.801064 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715dd16-0205-4b1b-b8fc-26d0a7709aae None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:04.801338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715dd16-0205-4b1b-b8fc-26d0a7709aae None admin] PATH_INFO: `/v3/roles/member` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:04.814675 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7715dd16-0205-4b1b-b8fc-26d0a7709aae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:24:04.820471 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-7715dd16-0205-4b1b-b8fc-26d0a7709aae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:24:04.820471 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:04.820471 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:04.820471 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:04.820471 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:04.820471 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:04.820471 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:04.820471 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:04.820471 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:04.820471 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:04.820471 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:04.820471 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:04.820471 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:24:04.820471 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:24:04.820471 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:24:04.820471 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:24:04.821360 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:24:04.821360 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:24:04.821360 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:24:04.821360 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:24:04.821360 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:04.821360 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:04.821360 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:04.821360 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:04.821360 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:04.821360 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:04.821360 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:04.821360 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:04.821360 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:04.821360 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:04.821360 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:04.821360 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:04.822137 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:04.822137 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:04.822137 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:04.822137 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:04.822137 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:24:04.822137 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:24:04.822137 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:24:04.822137 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:24:04.822137 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:24:04.822137 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:24:04.822137 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:24:04.822137 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:04.822137 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 196/399] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:24:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:04.827448 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ac77f-9a51-4911-b371-a1911116e42a None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:04.871885 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7715dd16-0205-4b1b-b8fc-26d0a7709aae None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:04.972825 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ac77f-9a51-4911-b371-a1911116e42a None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:05.007271 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:05.032906 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988ad54-ad7a-4640-a221-69730f04b44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:05.033915 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988ad54-ad7a-4640-a221-69730f04b44e None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:05.039044 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988ad54-ad7a-4640-a221-69730f04b44e None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:05.039567 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988ad54-ad7a-4640-a221-69730f04b44e None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:05.040486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2988ad54-ad7a-4640-a221-69730f04b44e None admin] RBAC: Adding query filter params (name=member) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:05.041920 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2988ad54-ad7a-4640-a221-69730f04b44e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:05.043704 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2988ad54-ad7a-4640-a221-69730f04b44e None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:05.056428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 204/400] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Aug 24 05:24:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:05.063957 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988ad54-ad7a-4640-a221-69730f04b44e None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:05.099414 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7715dd16-0205-4b1b-b8fc-26d0a7709aae None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:05.105258 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:05.167026 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58cece4-1fee-4a15-999e-e5bb0242b7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:05.167672 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58cece4-1fee-4a15-999e-e5bb0242b7ea None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:05.167958 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58cece4-1fee-4a15-999e-e5bb0242b7ea None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:05.168418 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58cece4-1fee-4a15-999e-e5bb0242b7ea None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:05.185862 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988ad54-ad7a-4640-a221-69730f04b44e None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:05.190985 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58cece4-1fee-4a15-999e-e5bb0242b7ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:24:05.196877 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-f58cece4-1fee-4a15-999e-e5bb0242b7ea None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:24:05.196877 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:05.196877 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:05.196877 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:05.196877 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:05.196877 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:05.196877 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:05.196877 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:05.196877 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:05.196877 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:05.196877 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:05.196877 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:05.196877 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:24:05.196877 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:24:05.196877 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:24:05.196877 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:24:05.197970 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:24:05.197970 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:24:05.197970 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:24:05.197970 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:24:05.197970 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:05.197970 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:05.197970 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:05.197970 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:05.197970 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:05.197970 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:05.197970 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:05.197970 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:05.197970 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:05.197970 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:05.197970 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:05.197970 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:05.198952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:05.198952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:05.198952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:05.198952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:05.198952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:24:05.198952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:24:05.198952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:24:05.198952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:24:05.198952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:24:05.198952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:24:05.198952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:24:05.198952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:05.200962 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 197/401] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Aug 24 05:24:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:24:05.202580 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eae8c7-0614-4bd4-acd7-a4ba61ae6967 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:05.203336 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eae8c7-0614-4bd4-acd7-a4ba61ae6967 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:05.203723 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eae8c7-0614-4bd4-acd7-a4ba61ae6967 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:05.205169 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 198/402] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:05.209375 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eae8c7-0614-4bd4-acd7-a4ba61ae6967 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:05.235428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed030b-3800-4c41-adcb-db74cc216f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:05.236135 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed030b-3800-4c41-adcb-db74cc216f32 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:05.236525 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed030b-3800-4c41-adcb-db74cc216f32 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:05.236913 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed030b-3800-4c41-adcb-db74cc216f32 None admin] PATH_INFO: `/v3/groups/abcb375861e44c97ba4d47a68e63ee18` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:05.238143 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fed030b-3800-4c41-adcb-db74cc216f32 None admin] RBAC: Authorizing `identity:get_group(group_id=abcb375861e44c97ba4d47a68e63ee18)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:05.239728 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fed030b-3800-4c41-adcb-db74cc216f32 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:05.245958 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 205/403] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Aug 24 05:24:05 2022] GET /identity/v3/groups/abcb375861e44c97ba4d47a68e63ee18 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:05.245958 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fec14f-16c4-49f4-8f6e-a86ce6b3ed7a None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:05.245958 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fec14f-16c4-49f4-8f6e-a86ce6b3ed7a None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:05.245958 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fec14f-16c4-49f4-8f6e-a86ce6b3ed7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:05.261753 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-24fec14f-16c4-49f4-8f6e-a86ce6b3ed7a None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:05.263064 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-24fec14f-16c4-49f4-8f6e-a86ce6b3ed7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:05.348504 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 206/404] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:24:05.349640 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fec14f-16c4-49f4-8f6e-a86ce6b3ed7a None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:05.351503 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eae8c7-0614-4bd4-acd7-a4ba61ae6967 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:05.373161 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:05.392736 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae914e7-604c-48da-93d9-47882e1ed144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:05.393636 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae914e7-604c-48da-93d9-47882e1ed144 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:05.394085 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae914e7-604c-48da-93d9-47882e1ed144 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:05.394757 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae914e7-604c-48da-93d9-47882e1ed144 None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:05.395606 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae914e7-604c-48da-93d9-47882e1ed144 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:05.396496 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae914e7-604c-48da-93d9-47882e1ed144 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:05.398041 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae914e7-604c-48da-93d9-47882e1ed144 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:05.409383 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 199/405] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:24:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:05.415228 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae914e7-604c-48da-93d9-47882e1ed144 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:05.422428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:05.527519 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae914e7-604c-48da-93d9-47882e1ed144 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:05.578388 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fec14f-16c4-49f4-8f6e-a86ce6b3ed7a None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:05.609766 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7bb39e-e175-4ebc-ab2c-2be7ada7eb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:05.610436 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7bb39e-e175-4ebc-ab2c-2be7ada7eb91 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:05.610726 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7bb39e-e175-4ebc-ab2c-2be7ada7eb91 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:05.611011 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7bb39e-e175-4ebc-ab2c-2be7ada7eb91 None admin] PATH_INFO: `/v3/groups/abcb375861e44c97ba4d47a68e63ee18` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:05.613273 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7bb39e-e175-4ebc-ab2c-2be7ada7eb91 None admin] RBAC: Authorizing `identity:get_group(group_id=abcb375861e44c97ba4d47a68e63ee18)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:05.614166 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dff374b-941b-4f8c-b7f7-20351dd9849d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:05.614775 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7bb39e-e175-4ebc-ab2c-2be7ada7eb91 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:05.614892 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff374b-941b-4f8c-b7f7-20351dd9849d None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:05.615075 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff374b-941b-4f8c-b7f7-20351dd9849d None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:05.615356 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff374b-941b-4f8c-b7f7-20351dd9849d None admin] PATH_INFO: `/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:05.616421 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 200/406] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Aug 24 05:24:05 2022] GET /identity/v3/groups/abcb375861e44c97ba4d47a68e63ee18 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:24:05.616658 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dff374b-941b-4f8c-b7f7-20351dd9849d None admin] RBAC: Authorizing `identity:get_project(project_id=a7f9d22fd79e4b1fa16a2f60fc6faec7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:05.618146 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dff374b-941b-4f8c-b7f7-20351dd9849d None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:05.619797 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 207/407] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:24:05 2022] GET /identity/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7 => generated 302 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:05.622352 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7bb39e-e175-4ebc-ab2c-2be7ada7eb91 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:05.626769 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dff374b-941b-4f8c-b7f7-20351dd9849d None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:05.718829 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:05.776981 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dff374b-941b-4f8c-b7f7-20351dd9849d None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:05.781202 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:05.787212 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7bb39e-e175-4ebc-ab2c-2be7ada7eb91 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:05.821037 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9856222-fa43-4200-bb47-4192a6dbb31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:05.821781 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9856222-fa43-4200-bb47-4192a6dbb31c None admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:05.822066 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9856222-fa43-4200-bb47-4192a6dbb31c None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:05.822401 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9856222-fa43-4200-bb47-4192a6dbb31c None admin] PATH_INFO: `/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7/groups/abcb375861e44c97ba4d47a68e63ee18/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:05.825086 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9856222-fa43-4200-bb47-4192a6dbb31c None admin] RBAC: Authorizing `identity:create_grant(project_id=a7f9d22fd79e4b1fa16a2f60fc6faec7, group_id=abcb375861e44c97ba4d47a68e63ee18, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:05.826537 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9856222-fa43-4200-bb47-4192a6dbb31c None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:05.837581 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75a8da-be41-48da-86c6-103760f75cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:05.838238 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75a8da-be41-48da-86c6-103760f75cce None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:05.838523 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75a8da-be41-48da-86c6-103760f75cce None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:05.838799 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75a8da-be41-48da-86c6-103760f75cce None admin] PATH_INFO: `/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:05.840076 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb75a8da-be41-48da-86c6-103760f75cce None admin] RBAC: Authorizing `identity:get_project(project_id=a7f9d22fd79e4b1fa16a2f60fc6faec7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:05.841704 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb75a8da-be41-48da-86c6-103760f75cce None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:05.848838 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 201/408] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:24:05 2022] GET /identity/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7 => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:24:05.848838 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75a8da-be41-48da-86c6-103760f75cce None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:05.952523 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75a8da-be41-48da-86c6-103760f75cce None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:05.971286 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:05.977088 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 208/409] 10.222.0.15 () {58 vars in 1442 bytes} [Wed Aug 24 05:24:05 2022] PUT /identity/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7/groups/abcb375861e44c97ba4d47a68e63ee18/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:24:05.978508 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9856222-fa43-4200-bb47-4192a6dbb31c None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:05.993943 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d8100-7f20-411a-8afa-698bab77ec36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:05.994590 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d8100-7f20-411a-8afa-698bab77ec36 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:05.995490 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d8100-7f20-411a-8afa-698bab77ec36 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:05.995795 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d8100-7f20-411a-8afa-698bab77ec36 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:06.010504 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d8100-7f20-411a-8afa-698bab77ec36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:24:06.018665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-a64d8100-7f20-411a-8afa-698bab77ec36 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:24:06.018665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:06.018665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:06.018665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:06.018665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:06.018665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:06.018665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:06.018665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:06.018665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:06.018665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:06.018665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:06.018665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:06.018665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:24:06.018665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:24:06.018665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:24:06.018665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:24:06.019692 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:24:06.019692 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:24:06.019692 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:24:06.019692 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:24:06.019692 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:06.019692 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:06.019692 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:06.019692 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:06.019692 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:06.019692 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:06.019692 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:06.019692 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:06.019692 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:06.019692 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:06.019692 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:06.019692 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:06.020558 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:06.020558 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:06.020558 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:06.020558 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:06.020558 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:24:06.020558 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:24:06.020558 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:24:06.020558 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:24:06.020558 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:24:06.020558 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:24:06.020558 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:24:06.020558 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:06.020558 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 202/410] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Aug 24 05:24:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:06.027094 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d8100-7f20-411a-8afa-698bab77ec36 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:06.103296 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9856222-fa43-4200-bb47-4192a6dbb31c None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:06.140482 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799a7a27-6de6-4bed-8296-620cec89d3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:06.141363 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a7a27-6de6-4bed-8296-620cec89d3ad None admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:06.141653 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a7a27-6de6-4bed-8296-620cec89d3ad None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:06.142000 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a7a27-6de6-4bed-8296-620cec89d3ad None admin] PATH_INFO: `/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7/groups/abcb375861e44c97ba4d47a68e63ee18/roles/0cdb52ffa00449399b27dbfcd383f488` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:06.144870 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799a7a27-6de6-4bed-8296-620cec89d3ad None admin] RBAC: Authorizing `identity:create_grant(project_id=a7f9d22fd79e4b1fa16a2f60fc6faec7, group_id=abcb375861e44c97ba4d47a68e63ee18, role_id=0cdb52ffa00449399b27dbfcd383f488)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:06.146102 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799a7a27-6de6-4bed-8296-620cec89d3ad None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:06.152426 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d8100-7f20-411a-8afa-698bab77ec36 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:06.195939 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c8175f-2e48-443b-be42-f89c83bfbf61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:06.196631 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c8175f-2e48-443b-be42-f89c83bfbf61 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:06.196918 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c8175f-2e48-443b-be42-f89c83bfbf61 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:06.197197 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c8175f-2e48-443b-be42-f89c83bfbf61 None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:06.197891 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c8175f-2e48-443b-be42-f89c83bfbf61 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:06.198210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c8175f-2e48-443b-be42-f89c83bfbf61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:06.199938 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c8175f-2e48-443b-be42-f89c83bfbf61 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:06.215355 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 209/411] 10.222.0.15 () {58 vars in 1442 bytes} [Wed Aug 24 05:24:05 2022] PUT /identity/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7/groups/abcb375861e44c97ba4d47a68e63ee18/roles/0cdb52ffa00449399b27dbfcd383f488 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:24:06.216586 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902bb4e3-311e-432b-ba3d-05acd1bfe350 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:06.217281 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902bb4e3-311e-432b-ba3d-05acd1bfe350 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:06.217920 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902bb4e3-311e-432b-ba3d-05acd1bfe350 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:06.219435 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 210/412] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:06.221558 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 203/413] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:24:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:24:06.226758 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59cdffe-4da5-4e65-af80-11e0e700e231 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:06.227051 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59cdffe-4da5-4e65-af80-11e0e700e231 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:06.227336 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59cdffe-4da5-4e65-af80-11e0e700e231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:06.228833 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c8175f-2e48-443b-be42-f89c83bfbf61 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:06.255533 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-f59cdffe-4da5-4e65-af80-11e0e700e231 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:06.256869 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-f59cdffe-4da5-4e65-af80-11e0e700e231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:06.279279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:06.291979 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:06.344310 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 211/414] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:24:06.383488 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c8175f-2e48-443b-be42-f89c83bfbf61 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:06.423369 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9040d8b-5dff-4ec1-80f7-0c21b36a121b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:06.424016 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9040d8b-5dff-4ec1-80f7-0c21b36a121b None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:06.424300 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9040d8b-5dff-4ec1-80f7-0c21b36a121b None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:06.424609 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9040d8b-5dff-4ec1-80f7-0c21b36a121b None admin] PATH_INFO: `/v3/groups/0371c18296e84248adae66efa0f5a092` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:06.425744 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9040d8b-5dff-4ec1-80f7-0c21b36a121b None admin] RBAC: Authorizing `identity:get_group(group_id=0371c18296e84248adae66efa0f5a092)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:06.427398 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9040d8b-5dff-4ec1-80f7-0c21b36a121b None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:06.429368 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 204/415] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Aug 24 05:24:06 2022] GET /identity/v3/groups/0371c18296e84248adae66efa0f5a092 => generated 229 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:06.435106 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59cdffe-4da5-4e65-af80-11e0e700e231 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:06.545899 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59cdffe-4da5-4e65-af80-11e0e700e231 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:06.587845 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045bc98-dfcb-49f8-830a-6ea18881b0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:06.588610 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045bc98-dfcb-49f8-830a-6ea18881b0f6 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:06.588894 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045bc98-dfcb-49f8-830a-6ea18881b0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:06.589176 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045bc98-dfcb-49f8-830a-6ea18881b0f6 None admin] PATH_INFO: `/v3/projects/617e47339cb94e20967707fe62db290d` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:06.593122 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2045bc98-dfcb-49f8-830a-6ea18881b0f6 None admin] RBAC: Authorizing `identity:get_project(project_id=617e47339cb94e20967707fe62db290d)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:06.595317 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2045bc98-dfcb-49f8-830a-6ea18881b0f6 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:06.597336 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 212/416] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:24:06 2022] GET /identity/v3/projects/617e47339cb94e20967707fe62db290d => generated 344 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:06.603459 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9040d8b-5dff-4ec1-80f7-0c21b36a121b None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:06.712320 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9040d8b-5dff-4ec1-80f7-0c21b36a121b None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:06.720132 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:06.748507 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4e6e16-ace2-46d5-83dd-6e0fb5bdc210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:06.749240 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e6e16-ace2-46d5-83dd-6e0fb5bdc210 None admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:06.749929 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e6e16-ace2-46d5-83dd-6e0fb5bdc210 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:06.750216 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e6e16-ace2-46d5-83dd-6e0fb5bdc210 None admin] PATH_INFO: `/v3/projects/617e47339cb94e20967707fe62db290d/groups/0371c18296e84248adae66efa0f5a092/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:06.752260 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4e6e16-ace2-46d5-83dd-6e0fb5bdc210 None admin] RBAC: Authorizing `identity:create_grant(project_id=617e47339cb94e20967707fe62db290d, group_id=0371c18296e84248adae66efa0f5a092, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:06.753790 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4e6e16-ace2-46d5-83dd-6e0fb5bdc210 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:06.801763 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045bc98-dfcb-49f8-830a-6ea18881b0f6 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:06.819212 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:06.884606 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 205/417] 10.222.0.15 () {58 vars in 1442 bytes} [Wed Aug 24 05:24:06 2022] PUT /identity/v3/projects/617e47339cb94e20967707fe62db290d/groups/0371c18296e84248adae66efa0f5a092/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:24:06.886248 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2e0b2-ed47-464d-8b48-61c9da925f85 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:06.886457 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2e0b2-ed47-464d-8b48-61c9da925f85 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:06.886713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2e0b2-ed47-464d-8b48-61c9da925f85 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:06.888208 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 206/418] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:06.905323 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5247bb1-0ed2-46fb-afb2-37e615dd525b None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:06.905859 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5247bb1-0ed2-46fb-afb2-37e615dd525b None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:06.906134 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5247bb1-0ed2-46fb-afb2-37e615dd525b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:06.911855 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045bc98-dfcb-49f8-830a-6ea18881b0f6 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:06.949581 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-a5247bb1-0ed2-46fb-afb2-37e615dd525b None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:06.950816 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-a5247bb1-0ed2-46fb-afb2-37e615dd525b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:06.954290 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b538d1d-1bf7-4b32-9197-90387b3c7fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:06.955343 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b538d1d-1bf7-4b32-9197-90387b3c7fa0 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:06.955903 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b538d1d-1bf7-4b32-9197-90387b3c7fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:06.956371 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b538d1d-1bf7-4b32-9197-90387b3c7fa0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:06.971398 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b538d1d-1bf7-4b32-9197-90387b3c7fa0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:24:06.977101 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-2b538d1d-1bf7-4b32-9197-90387b3c7fa0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:24:06.977101 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:06.977101 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:06.977101 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:06.977101 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:06.977101 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:06.977101 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:06.977101 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:06.977101 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:06.977101 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:06.977101 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:06.977101 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:06.977101 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:24:06.977101 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:24:06.977101 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:24:06.977101 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:24:06.978038 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:24:06.978038 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:24:06.978038 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:24:06.978038 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:24:06.978038 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:06.978038 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:06.978038 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:06.978038 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:06.978038 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:06.978038 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:06.978038 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:06.978038 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:06.978038 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:06.978038 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:06.978038 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:06.978038 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:06.978856 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:06.978856 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:06.978856 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:06.978856 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:06.978856 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:24:06.978856 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:24:06.978856 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:24:06.978856 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:24:06.978856 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:24:06.978856 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:24:06.978856 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:24:06.978856 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:06.981785 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 213/419] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:24:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:24:06.988635 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b538d1d-1bf7-4b32-9197-90387b3c7fa0 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:07.089733 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:07.100452 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 207/420] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:24:07.118222 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b538d1d-1bf7-4b32-9197-90387b3c7fa0 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:07.154919 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee64252b-ee1d-4221-a54d-d3a4453df5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:07.156551 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64252b-ee1d-4221-a54d-d3a4453df5e2 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:07.156983 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64252b-ee1d-4221-a54d-d3a4453df5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:07.157263 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64252b-ee1d-4221-a54d-d3a4453df5e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:07.157995 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee64252b-ee1d-4221-a54d-d3a4453df5e2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:07.158305 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee64252b-ee1d-4221-a54d-d3a4453df5e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:07.159877 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee64252b-ee1d-4221-a54d-d3a4453df5e2 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:07.168983 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 214/421] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Aug 24 05:24:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:07.175120 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5247bb1-0ed2-46fb-afb2-37e615dd525b None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:07.325410 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:07.332375 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5247bb1-0ed2-46fb-afb2-37e615dd525b None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:07.381465 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245cbe1-7bc0-4ff0-b8b2-8e8ba27dce07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:07.382087 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245cbe1-7bc0-4ff0-b8b2-8e8ba27dce07 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:07.382373 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245cbe1-7bc0-4ff0-b8b2-8e8ba27dce07 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:07.382647 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245cbe1-7bc0-4ff0-b8b2-8e8ba27dce07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:07.474844 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee64252b-ee1d-4221-a54d-d3a4453df5e2 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:07.486965 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9245cbe1-7bc0-4ff0-b8b2-8e8ba27dce07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:07.486965 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9245cbe1-7bc0-4ff0-b8b2-8e8ba27dce07 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:07.592872 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:07.604800 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 208/422] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:24:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:24:07.608106 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245cbe1-7bc0-4ff0-b8b2-8e8ba27dce07 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:07.716999 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee64252b-ee1d-4221-a54d-d3a4453df5e2 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:07.734933 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245cbe1-7bc0-4ff0-b8b2-8e8ba27dce07 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:07.771565 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459088ab-a0ce-407f-aae5-f220ab4c0ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:07.772230 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459088ab-a0ce-407f-aae5-f220ab4c0ccc None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:07.772623 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459088ab-a0ce-407f-aae5-f220ab4c0ccc None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:07.772915 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459088ab-a0ce-407f-aae5-f220ab4c0ccc None admin] PATH_INFO: `/v3/projects/4c7185cb4791495281d3a16293793beb` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:07.774157 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459088ab-a0ce-407f-aae5-f220ab4c0ccc None admin] RBAC: Authorizing `identity:get_project(project_id=4c7185cb4791495281d3a16293793beb)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:07.779284 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459088ab-a0ce-407f-aae5-f220ab4c0ccc None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:07.779284 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 209/423] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:24:07 2022] GET /identity/v3/projects/4c7185cb4791495281d3a16293793beb => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:24:07.785255 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459088ab-a0ce-407f-aae5-f220ab4c0ccc None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:07.845810 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69775bf6-e40f-4459-be41-d3acbc67034a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:07.846822 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69775bf6-e40f-4459-be41-d3acbc67034a None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:07.847303 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69775bf6-e40f-4459-be41-d3acbc67034a None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:07.847772 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69775bf6-e40f-4459-be41-d3acbc67034a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:07.860766 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:07.864869 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:07.893041 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69775bf6-e40f-4459-be41-d3acbc67034a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:24:07.895976 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459088ab-a0ce-407f-aae5-f220ab4c0ccc None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:07.898319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-69775bf6-e40f-4459-be41-d3acbc67034a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:24:07.898319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:07.898319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:07.898319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:07.898319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:07.898319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:07.898319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:07.898319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:07.898319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:07.898319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:07.898319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:07.898319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:07.898319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:24:07.898319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:24:07.898319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:24:07.898319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:24:07.899189 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:24:07.899189 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:24:07.899189 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:24:07.899189 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:24:07.899189 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:07.899189 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:07.899189 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:07.899189 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:07.899189 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:07.899189 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:07.899189 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:07.899189 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:07.899189 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:07.899189 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:07.899189 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:07.899189 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:07.899976 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:07.899976 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:07.899976 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:07.899976 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:07.899976 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:24:07.899976 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:24:07.899976 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:24:07.899976 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:24:07.899976 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:24:07.899976 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:24:07.899976 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:24:07.899976 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:07.908571 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 215/424] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Aug 24 05:24:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 434 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:07.909689 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69775bf6-e40f-4459-be41-d3acbc67034a None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:07.935270 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810dcefd-039d-42e3-b65c-d56e5186f266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:07.936060 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810dcefd-039d-42e3-b65c-d56e5186f266 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:07.936427 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810dcefd-039d-42e3-b65c-d56e5186f266 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:07.936792 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810dcefd-039d-42e3-b65c-d56e5186f266 None admin] PATH_INFO: `/v3/groups/abcb375861e44c97ba4d47a68e63ee18` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:07.939290 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810dcefd-039d-42e3-b65c-d56e5186f266 None admin] RBAC: Authorizing `identity:get_group(group_id=abcb375861e44c97ba4d47a68e63ee18)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:07.941066 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810dcefd-039d-42e3-b65c-d56e5186f266 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:07.943413 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 210/425] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Aug 24 05:24:07 2022] GET /identity/v3/groups/abcb375861e44c97ba4d47a68e63ee18 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:07.949643 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810dcefd-039d-42e3-b65c-d56e5186f266 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:08.065081 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810dcefd-039d-42e3-b65c-d56e5186f266 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:08.102951 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acabaab9-db71-4f4b-9faf-288883c4beee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:08.103561 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabaab9-db71-4f4b-9faf-288883c4beee None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:08.103914 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabaab9-db71-4f4b-9faf-288883c4beee None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:08.104282 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabaab9-db71-4f4b-9faf-288883c4beee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:08.105494 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acabaab9-db71-4f4b-9faf-288883c4beee None admin] RBAC: Adding query filter params (group.id=abcb375861e44c97ba4d47a68e63ee18, role.id=87862cd6eaeb4d5fae089a801aac4f87, scope.project.id=4c7185cb4791495281d3a16293793beb) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:08.105875 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acabaab9-db71-4f4b-9faf-288883c4beee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:08.107547 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acabaab9-db71-4f4b-9faf-288883c4beee None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:08.114744 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 211/426] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Aug 24 05:24:07 2022] GET /identity/v3/role_assignments?group.id=abcb375861e44c97ba4d47a68e63ee18&scope.project.id=4c7185cb4791495281d3a16293793beb&role.id=87862cd6eaeb4d5fae089a801aac4f87 => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:08.115027 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:08.223377 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69775bf6-e40f-4459-be41-d3acbc67034a None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:08.289745 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edea45f-228e-433d-b643-23be4d1fa4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:08.290701 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edea45f-228e-433d-b643-23be4d1fa4d1 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:08.291161 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edea45f-228e-433d-b643-23be4d1fa4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:08.291610 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edea45f-228e-433d-b643-23be4d1fa4d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:08.292645 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edea45f-228e-433d-b643-23be4d1fa4d1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:08.293146 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edea45f-228e-433d-b643-23be4d1fa4d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:08.294724 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edea45f-228e-433d-b643-23be4d1fa4d1 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:08.304776 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 216/427] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:24:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:08.310431 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acabaab9-db71-4f4b-9faf-288883c4beee None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:08.333054 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb601d47-cf80-4ac1-9b4c-9e152eea7fd4 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:08.333620 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb601d47-cf80-4ac1-9b4c-9e152eea7fd4 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:08.334135 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb601d47-cf80-4ac1-9b4c-9e152eea7fd4 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:08.335902 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 217/428] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:24:08.346425 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6433df-6e03-4724-920f-a06efac2745f None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:08.347438 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6433df-6e03-4724-920f-a06efac2745f None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:08.347901 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6433df-6e03-4724-920f-a06efac2745f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:08.372282 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-ba6433df-6e03-4724-920f-a06efac2745f None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:08.373796 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-ba6433df-6e03-4724-920f-a06efac2745f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:08.438704 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:08.447978 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acabaab9-db71-4f4b-9faf-288883c4beee None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:08.457029 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 218/429] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:24:08.496816 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271219d-4f18-4893-ab00-4e32bab0625b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:08.498756 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271219d-4f18-4893-ab00-4e32bab0625b None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:08.499225 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271219d-4f18-4893-ab00-4e32bab0625b None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:08.500669 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271219d-4f18-4893-ab00-4e32bab0625b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:08.576042 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:08.584162 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9271219d-4f18-4893-ab00-4e32bab0625b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:08.590154 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9271219d-4f18-4893-ab00-4e32bab0625b None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:08.708450 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 212/430] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:24:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 05:24:08.715720 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6433df-6e03-4724-920f-a06efac2745f None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:08.778530 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fdd554-59e3-4976-9c30-b3adb1b749a0 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:08.778839 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fdd554-59e3-4976-9c30-b3adb1b749a0 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:08.779094 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fdd554-59e3-4976-9c30-b3adb1b749a0 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:08.781036 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 213/431] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:08.788959 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244c8c4-8872-4c96-8c4a-277efb77e4bd None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:08.789368 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244c8c4-8872-4c96-8c4a-277efb77e4bd None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:08.789674 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244c8c4-8872-4c96-8c4a-277efb77e4bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:08.813818 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-f244c8c4-8872-4c96-8c4a-277efb77e4bd None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:08.815080 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-f244c8c4-8872-4c96-8c4a-277efb77e4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:08.832943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6433df-6e03-4724-920f-a06efac2745f None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:08.887877 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6429bb53-d097-4523-83bd-04125fe2993e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:08.889336 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429bb53-d097-4523-83bd-04125fe2993e None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:08.889336 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429bb53-d097-4523-83bd-04125fe2993e None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:08.889667 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429bb53-d097-4523-83bd-04125fe2993e None admin] PATH_INFO: `/v3/projects/4c7185cb4791495281d3a16293793beb` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:08.891991 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6429bb53-d097-4523-83bd-04125fe2993e None admin] RBAC: Authorizing `identity:get_project(project_id=4c7185cb4791495281d3a16293793beb)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:08.894403 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6429bb53-d097-4523-83bd-04125fe2993e None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:08.896370 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 219/432] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:24:08 2022] GET /identity/v3/projects/4c7185cb4791495281d3a16293793beb => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:08.896994 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6429bb53-d097-4523-83bd-04125fe2993e None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:08.928248 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 214/433] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:24:08.928952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f244c8c4-8872-4c96-8c4a-277efb77e4bd None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:08.971130 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:08.975537 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:09.025317 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6429bb53-d097-4523-83bd-04125fe2993e None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:09.053372 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f244c8c4-8872-4c96-8c4a-277efb77e4bd None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:09.062408 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f35913-9eb1-4a24-9711-dea0a967dea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:09.063398 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f35913-9eb1-4a24-9711-dea0a967dea4 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:09.063882 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f35913-9eb1-4a24-9711-dea0a967dea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:09.064351 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f35913-9eb1-4a24-9711-dea0a967dea4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:09.075650 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f35913-9eb1-4a24-9711-dea0a967dea4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:24:09.082007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-80f35913-9eb1-4a24-9711-dea0a967dea4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:24:09.082007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:09.082007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:09.082007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:09.082007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:09.082007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:09.082007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:09.082007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:09.082007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:09.082007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:09.082007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:09.082007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:09.082007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:24:09.082007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:24:09.082007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:24:09.082007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:24:09.082907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:24:09.082907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:24:09.082907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:24:09.082907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:24:09.082907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:09.082907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:09.082907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:09.082907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:09.082907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:09.082907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:09.082907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:09.082907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:09.082907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:09.082907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:09.082907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:09.082907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:09.083700 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:09.083700 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:09.083700 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:09.083700 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:09.083700 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:24:09.083700 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:24:09.083700 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:24:09.083700 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:24:09.083700 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:24:09.083700 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:24:09.083700 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:24:09.083700 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:09.090676 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 220/434] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:24:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:09.092432 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f35913-9eb1-4a24-9711-dea0a967dea4 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:09.094839 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e2d656-5a29-45a1-a023-03f917708b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:09.095605 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2d656-5a29-45a1-a023-03f917708b56 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:09.096020 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2d656-5a29-45a1-a023-03f917708b56 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:09.096370 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2d656-5a29-45a1-a023-03f917708b56 None admin] PATH_INFO: `/v3/groups/abcb375861e44c97ba4d47a68e63ee18` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:09.097640 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e2d656-5a29-45a1-a023-03f917708b56 None admin] RBAC: Authorizing `identity:get_group(group_id=abcb375861e44c97ba4d47a68e63ee18)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:09.099156 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e2d656-5a29-45a1-a023-03f917708b56 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:09.101230 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 215/435] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Aug 24 05:24:08 2022] GET /identity/v3/groups/abcb375861e44c97ba4d47a68e63ee18 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:09.106525 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e2d656-5a29-45a1-a023-03f917708b56 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:09.224000 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f35913-9eb1-4a24-9711-dea0a967dea4 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:09.224817 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:09.231600 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e2d656-5a29-45a1-a023-03f917708b56 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:09.241356 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:09.259841 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b67779-83f0-4a26-b7a5-0244eac1cc89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:09.260486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b67779-83f0-4a26-b7a5-0244eac1cc89 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:09.260771 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b67779-83f0-4a26-b7a5-0244eac1cc89 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:09.261049 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b67779-83f0-4a26-b7a5-0244eac1cc89 None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:09.261714 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b67779-83f0-4a26-b7a5-0244eac1cc89 None admin] RBAC: Adding query filter params (name=member) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:09.262019 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b67779-83f0-4a26-b7a5-0244eac1cc89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:09.263422 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b67779-83f0-4a26-b7a5-0244eac1cc89 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:09.270361 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d5609-ee29-4cf9-87a1-9dc8cc1563c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:09.276960 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d5609-ee29-4cf9-87a1-9dc8cc1563c5 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:09.276960 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d5609-ee29-4cf9-87a1-9dc8cc1563c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:09.276960 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d5609-ee29-4cf9-87a1-9dc8cc1563c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:09.276960 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116d5609-ee29-4cf9-87a1-9dc8cc1563c5 None admin] RBAC: Adding query filter params (group.id=abcb375861e44c97ba4d47a68e63ee18, role.id=0cdb52ffa00449399b27dbfcd383f488, scope.project.id=4c7185cb4791495281d3a16293793beb) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:09.276960 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116d5609-ee29-4cf9-87a1-9dc8cc1563c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:09.277317 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 221/436] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Aug 24 05:24:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:24:09.277317 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b67779-83f0-4a26-b7a5-0244eac1cc89 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:09.277441 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116d5609-ee29-4cf9-87a1-9dc8cc1563c5 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:09.282937 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 216/437] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Aug 24 05:24:09 2022] GET /identity/v3/role_assignments?group.id=abcb375861e44c97ba4d47a68e63ee18&scope.project.id=4c7185cb4791495281d3a16293793beb&role.id=0cdb52ffa00449399b27dbfcd383f488 => generated 611 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:09.283902 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d5609-ee29-4cf9-87a1-9dc8cc1563c5 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:09.415660 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b67779-83f0-4a26-b7a5-0244eac1cc89 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:09.419800 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d5609-ee29-4cf9-87a1-9dc8cc1563c5 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:09.458675 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9417c5d3-98b3-476f-ad47-deb9885fe491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:09.459332 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9417c5d3-98b3-476f-ad47-deb9885fe491 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:09.459614 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9417c5d3-98b3-476f-ad47-deb9885fe491 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:09.459886 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9417c5d3-98b3-476f-ad47-deb9885fe491 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:09.467586 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca9930-2d82-4094-944d-0b5a8e587302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:09.467586 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca9930-2d82-4094-944d-0b5a8e587302 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:09.467586 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca9930-2d82-4094-944d-0b5a8e587302 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:09.467586 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca9930-2d82-4094-944d-0b5a8e587302 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:09.477115 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9417c5d3-98b3-476f-ad47-deb9885fe491 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:24:09.482173 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-9417c5d3-98b3-476f-ad47-deb9885fe491 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:24:09.482173 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:09.482173 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:09.482173 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:09.482173 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:09.482173 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:09.482173 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:09.482173 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:09.482173 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:09.482173 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:09.482173 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:09.482173 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:09.482173 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:24:09.482173 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:24:09.482173 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:24:09.482173 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:24:09.483263 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:24:09.483263 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:24:09.483263 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:24:09.483263 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:24:09.483263 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:09.483263 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:09.483263 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:09.483263 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:09.483263 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:09.483263 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:09.483263 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:09.483263 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:09.483263 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:09.483263 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:09.483263 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:09.483263 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:09.484956 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:09.484956 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:09.484956 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:09.484956 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:09.484956 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:24:09.484956 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:24:09.484956 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:24:09.484956 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:24:09.484956 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:24:09.484956 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:24:09.484956 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:24:09.484956 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:09.484956 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 222/438] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Aug 24 05:24:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:09.487075 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10631419-0e20-4576-8c9a-bf0c9b6d113c None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:09.487075 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10631419-0e20-4576-8c9a-bf0c9b6d113c None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:09.487075 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10631419-0e20-4576-8c9a-bf0c9b6d113c None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:09.488187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 223/439] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:09.493196 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10631419-0e20-4576-8c9a-bf0c9b6d113c None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:09.505262 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:09.544515 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:09.564967 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cca9930-2d82-4094-944d-0b5a8e587302 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:09.566451 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cca9930-2d82-4094-944d-0b5a8e587302 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:09.623870 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10631419-0e20-4576-8c9a-bf0c9b6d113c None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:09.667729 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8164b0-b61a-486e-8966-249fe241f261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:09.668352 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8164b0-b61a-486e-8966-249fe241f261 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:09.668671 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8164b0-b61a-486e-8966-249fe241f261 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:09.668948 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8164b0-b61a-486e-8966-249fe241f261 None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:09.669616 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8164b0-b61a-486e-8966-249fe241f261 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:09.669924 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8164b0-b61a-486e-8966-249fe241f261 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:09.671846 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8164b0-b61a-486e-8966-249fe241f261 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:09.679866 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 224/440] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:24:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:09.681072 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a80b4-2259-4877-afb0-58ec6a21378a None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:09.681359 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a80b4-2259-4877-afb0-58ec6a21378a None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:09.681716 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a80b4-2259-4877-afb0-58ec6a21378a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:09.685915 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 217/441] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:24:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:24:09.686674 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca9930-2d82-4094-944d-0b5a8e587302 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:09.700366 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-d48a80b4-2259-4877-afb0-58ec6a21378a None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:09.701608 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-d48a80b4-2259-4877-afb0-58ec6a21378a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:09.746218 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:09.769808 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 225/442] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:24:09.781281 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a80b4-2259-4877-afb0-58ec6a21378a None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:09.792136 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca9930-2d82-4094-944d-0b5a8e587302 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:09.814424 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:09.831429 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94750857-1ebe-429a-9141-4500b4081aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:09.832038 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94750857-1ebe-429a-9141-4500b4081aa8 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:09.832322 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94750857-1ebe-429a-9141-4500b4081aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:09.832635 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94750857-1ebe-429a-9141-4500b4081aa8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:09.888645 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a80b4-2259-4877-afb0-58ec6a21378a None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:09.903767 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94750857-1ebe-429a-9141-4500b4081aa8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:09.905267 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94750857-1ebe-429a-9141-4500b4081aa8 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:09.925686 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1020624-3189-40f9-a208-630c1c401664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:09.926328 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1020624-3189-40f9-a208-630c1c401664 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:09.926609 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1020624-3189-40f9-a208-630c1c401664 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:09.926888 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1020624-3189-40f9-a208-630c1c401664 None admin] PATH_INFO: `/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:09.928026 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1020624-3189-40f9-a208-630c1c401664 None admin] RBAC: Authorizing `identity:get_project(project_id=a7f9d22fd79e4b1fa16a2f60fc6faec7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:09.929502 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1020624-3189-40f9-a208-630c1c401664 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:09.931239 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 226/443] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:24:09 2022] GET /identity/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:09.937674 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1020624-3189-40f9-a208-630c1c401664 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:09.969658 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:10.016982 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 218/444] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:24:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:24:10.022747 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94750857-1ebe-429a-9141-4500b4081aa8 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:10.045530 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:10.049688 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1020624-3189-40f9-a208-630c1c401664 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:10.084215 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0d193-1132-4133-9e42-12dd9ff6ddd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:10.084875 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0d193-1132-4133-9e42-12dd9ff6ddd6 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:10.085158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0d193-1132-4133-9e42-12dd9ff6ddd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:10.085435 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0d193-1132-4133-9e42-12dd9ff6ddd6 None admin] PATH_INFO: `/v3/groups/abcb375861e44c97ba4d47a68e63ee18` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:10.086613 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe0d193-1132-4133-9e42-12dd9ff6ddd6 None admin] RBAC: Authorizing `identity:get_group(group_id=abcb375861e44c97ba4d47a68e63ee18)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:10.088577 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe0d193-1132-4133-9e42-12dd9ff6ddd6 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:10.096353 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 227/445] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Aug 24 05:24:09 2022] GET /identity/v3/groups/abcb375861e44c97ba4d47a68e63ee18 => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:24:10.096353 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0d193-1132-4133-9e42-12dd9ff6ddd6 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:10.130304 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94750857-1ebe-429a-9141-4500b4081aa8 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:10.166589 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0955b9-7e7d-4460-bc37-248ddee15f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:10.167222 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0955b9-7e7d-4460-bc37-248ddee15f98 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:10.167511 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0955b9-7e7d-4460-bc37-248ddee15f98 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:10.167790 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0955b9-7e7d-4460-bc37-248ddee15f98 None admin] PATH_INFO: `/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:10.168948 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0955b9-7e7d-4460-bc37-248ddee15f98 None admin] RBAC: Authorizing `identity:get_project(project_id=a7f9d22fd79e4b1fa16a2f60fc6faec7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:10.170382 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0955b9-7e7d-4460-bc37-248ddee15f98 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:10.172649 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 219/446] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:24:10 2022] GET /identity/v3/projects/a7f9d22fd79e4b1fa16a2f60fc6faec7 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:24:10.173422 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0955b9-7e7d-4460-bc37-248ddee15f98 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:10.199449 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0d193-1132-4133-9e42-12dd9ff6ddd6 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:10.205126 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:10.234186 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4715c2-5256-4903-81b0-d5ad4f30b058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:10.235014 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4715c2-5256-4903-81b0-d5ad4f30b058 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:10.235296 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4715c2-5256-4903-81b0-d5ad4f30b058 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:10.235571 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4715c2-5256-4903-81b0-d5ad4f30b058 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:10.246096 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4715c2-5256-4903-81b0-d5ad4f30b058 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:24:10.251902 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-2d4715c2-5256-4903-81b0-d5ad4f30b058 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:24:10.251902 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:10.251902 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:10.251902 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:10.251902 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:10.251902 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:10.251902 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:10.251902 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:10.251902 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:10.251902 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:10.251902 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:10.251902 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:10.251902 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:24:10.251902 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:24:10.251902 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:24:10.251902 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:24:10.252992 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:24:10.252992 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:24:10.252992 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:24:10.252992 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:24:10.252992 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:10.252992 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:10.252992 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:10.252992 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:10.252992 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:10.252992 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:10.252992 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:10.252992 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:10.252992 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:10.252992 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:10.252992 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:10.252992 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:10.253775 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:10.253775 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:10.253775 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:10.253775 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:10.253775 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:24:10.253775 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:24:10.253775 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:24:10.253775 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:24:10.253775 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:24:10.253775 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:24:10.253775 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:24:10.253775 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:10.253775 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 228/447] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Aug 24 05:24:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:10.254504 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4715c2-5256-4903-81b0-d5ad4f30b058 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:10.284743 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:10.287438 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0955b9-7e7d-4460-bc37-248ddee15f98 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:10.323352 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a2015-1258-4f99-940f-499e9e29d483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:10.323957 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a2015-1258-4f99-940f-499e9e29d483 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:10.324237 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a2015-1258-4f99-940f-499e9e29d483 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:10.324557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a2015-1258-4f99-940f-499e9e29d483 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:10.325285 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190a2015-1258-4f99-940f-499e9e29d483 None admin] RBAC: Adding query filter params (group.id=abcb375861e44c97ba4d47a68e63ee18, role.id=87862cd6eaeb4d5fae089a801aac4f87, scope.project.id=a7f9d22fd79e4b1fa16a2f60fc6faec7) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:10.325601 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190a2015-1258-4f99-940f-499e9e29d483 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:10.327062 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190a2015-1258-4f99-940f-499e9e29d483 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:10.334129 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 220/448] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Aug 24 05:24:10 2022] GET /identity/v3/role_assignments?group.id=abcb375861e44c97ba4d47a68e63ee18&scope.project.id=a7f9d22fd79e4b1fa16a2f60fc6faec7&role.id=87862cd6eaeb4d5fae089a801aac4f87 => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:10.334928 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a2015-1258-4f99-940f-499e9e29d483 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:10.363445 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4715c2-5256-4903-81b0-d5ad4f30b058 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:10.406166 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f5be8-6797-41e6-8261-a622fc6ad1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:10.406820 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f5be8-6797-41e6-8261-a622fc6ad1c9 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:10.407106 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f5be8-6797-41e6-8261-a622fc6ad1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:10.407384 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f5be8-6797-41e6-8261-a622fc6ad1c9 None admin] PATH_INFO: `/v3/groups/abcb375861e44c97ba4d47a68e63ee18` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:10.408606 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3f5be8-6797-41e6-8261-a622fc6ad1c9 None admin] RBAC: Authorizing `identity:get_group(group_id=abcb375861e44c97ba4d47a68e63ee18)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:10.410083 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3f5be8-6797-41e6-8261-a622fc6ad1c9 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:10.411749 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 229/449] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Aug 24 05:24:10 2022] GET /identity/v3/groups/abcb375861e44c97ba4d47a68e63ee18 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:10.418006 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f5be8-6797-41e6-8261-a622fc6ad1c9 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:10.446472 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:10.459076 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a2015-1258-4f99-940f-499e9e29d483 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:10.497646 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3591ac4d-8c46-4c33-a2af-95b68a0c5a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:10.498263 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591ac4d-8c46-4c33-a2af-95b68a0c5a16 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:10.499484 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591ac4d-8c46-4c33-a2af-95b68a0c5a16 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:10.499484 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591ac4d-8c46-4c33-a2af-95b68a0c5a16 None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:10.499669 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3591ac4d-8c46-4c33-a2af-95b68a0c5a16 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:10.499957 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3591ac4d-8c46-4c33-a2af-95b68a0c5a16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:10.501559 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3591ac4d-8c46-4c33-a2af-95b68a0c5a16 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:10.512048 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 221/450] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:24:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:10.518565 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3591ac4d-8c46-4c33-a2af-95b68a0c5a16 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:10.533096 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:10.546962 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f5be8-6797-41e6-8261-a622fc6ad1c9 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:10.589914 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd7c660-3604-435e-b9ea-53da4d32041d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:10.590534 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd7c660-3604-435e-b9ea-53da4d32041d None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:10.590819 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd7c660-3604-435e-b9ea-53da4d32041d None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:10.591097 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd7c660-3604-435e-b9ea-53da4d32041d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:10.592211 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd7c660-3604-435e-b9ea-53da4d32041d None admin] RBAC: Adding query filter params (group.id=abcb375861e44c97ba4d47a68e63ee18, role.id=0cdb52ffa00449399b27dbfcd383f488, scope.project.id=a7f9d22fd79e4b1fa16a2f60fc6faec7) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:10.592211 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd7c660-3604-435e-b9ea-53da4d32041d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:10.593874 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd7c660-3604-435e-b9ea-53da4d32041d None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:10.600004 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 230/451] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Aug 24 05:24:10 2022] GET /identity/v3/role_assignments?group.id=abcb375861e44c97ba4d47a68e63ee18&scope.project.id=a7f9d22fd79e4b1fa16a2f60fc6faec7&role.id=0cdb52ffa00449399b27dbfcd383f488 => generated 611 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:24:10.657649 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3591ac4d-8c46-4c33-a2af-95b68a0c5a16 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:10.701321 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d9a29-fa0c-430b-bdfb-15557213486d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:10.702032 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d9a29-fa0c-430b-bdfb-15557213486d None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:10.702315 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d9a29-fa0c-430b-bdfb-15557213486d None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:10.703164 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d9a29-fa0c-430b-bdfb-15557213486d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:10.729835 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:10.779545 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6d9a29-fa0c-430b-bdfb-15557213486d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:10.781802 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6d9a29-fa0c-430b-bdfb-15557213486d None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:10.885407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 222/452] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:24:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 05:24:10.891702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd7c660-3604-435e-b9ea-53da4d32041d None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:10.994086 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd7c660-3604-435e-b9ea-53da4d32041d None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:11.025688 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db32a8b-36dc-4a08-b744-422a9615a208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:11.026308 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db32a8b-36dc-4a08-b744-422a9615a208 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:11.026584 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db32a8b-36dc-4a08-b744-422a9615a208 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:11.026858 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db32a8b-36dc-4a08-b744-422a9615a208 None admin] PATH_INFO: `/v3/projects/617e47339cb94e20967707fe62db290d` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:11.027961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db32a8b-36dc-4a08-b744-422a9615a208 None admin] RBAC: Authorizing `identity:get_project(project_id=617e47339cb94e20967707fe62db290d)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:11.029424 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db32a8b-36dc-4a08-b744-422a9615a208 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:11.031565 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 231/453] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:24:10 2022] GET /identity/v3/projects/617e47339cb94e20967707fe62db290d => generated 344 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:11.037351 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d9a29-fa0c-430b-bdfb-15557213486d None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:11.089083 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:11.137446 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d9a29-fa0c-430b-bdfb-15557213486d None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:11.171162 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb763ee2-7458-4d73-aeff-f5fdc8a3952f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:11.171783 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb763ee2-7458-4d73-aeff-f5fdc8a3952f None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:11.172069 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb763ee2-7458-4d73-aeff-f5fdc8a3952f None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:11.172347 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb763ee2-7458-4d73-aeff-f5fdc8a3952f None admin] PATH_INFO: `/v3/groups/0371c18296e84248adae66efa0f5a092` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:11.173589 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb763ee2-7458-4d73-aeff-f5fdc8a3952f None admin] RBAC: Authorizing `identity:get_group(group_id=0371c18296e84248adae66efa0f5a092)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:11.175024 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb763ee2-7458-4d73-aeff-f5fdc8a3952f None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:11.176629 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 223/454] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Aug 24 05:24:11 2022] GET /identity/v3/groups/0371c18296e84248adae66efa0f5a092 => generated 229 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:11.182698 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db32a8b-36dc-4a08-b744-422a9615a208 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:11.213308 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:11.284610 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db32a8b-36dc-4a08-b744-422a9615a208 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:11.317402 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db4b22e-b4a7-4282-a9ff-51de1d2494cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:11.317992 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4b22e-b4a7-4282-a9ff-51de1d2494cc None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:11.318270 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4b22e-b4a7-4282-a9ff-51de1d2494cc None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:11.318542 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4b22e-b4a7-4282-a9ff-51de1d2494cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:11.319279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db4b22e-b4a7-4282-a9ff-51de1d2494cc None admin] RBAC: Adding query filter params (group.id=0371c18296e84248adae66efa0f5a092, role.id=e013ad4d436f4cb0a9bf7debb7d13dc7, scope.project.id=617e47339cb94e20967707fe62db290d) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:11.319585 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db4b22e-b4a7-4282-a9ff-51de1d2494cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:11.321100 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db4b22e-b4a7-4282-a9ff-51de1d2494cc None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:11.326607 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 232/455] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Aug 24 05:24:11 2022] GET /identity/v3/role_assignments?group.id=0371c18296e84248adae66efa0f5a092&scope.project.id=617e47339cb94e20967707fe62db290d&role.id=e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:13.857732 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ce5c0-a57d-4258-901f-77a85ebf0570 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:13.858768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ce5c0-a57d-4258-901f-77a85ebf0570 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:13.859308 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ce5c0-a57d-4258-901f-77a85ebf0570 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:13.861335 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 224/456] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:13.866687 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e18f2-d4dd-4723-b22d-dad40029e911 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:13.867541 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e18f2-d4dd-4723-b22d-dad40029e911 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:13.868018 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e18f2-d4dd-4723-b22d-dad40029e911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:13.893369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-670e18f2-d4dd-4723-b22d-dad40029e911 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:13.894885 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-670e18f2-d4dd-4723-b22d-dad40029e911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:13.979100 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fcee8-1e15-4ba4-aeaf-14da97c92790 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:13.979738 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fcee8-1e15-4ba4-aeaf-14da97c92790 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:13.979955 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 233/457] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:24:13.980447 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fcee8-1e15-4ba4-aeaf-14da97c92790 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:13.983081 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 225/458] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:13.987543 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce281e-9010-4ae8-aa2b-3c430fa2a181 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:13.987832 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce281e-9010-4ae8-aa2b-3c430fa2a181 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:13.988145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce281e-9010-4ae8-aa2b-3c430fa2a181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:14.019194 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-33ce281e-9010-4ae8-aa2b-3c430fa2a181 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:14.020389 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-33ce281e-9010-4ae8-aa2b-3c430fa2a181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:14.043772 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:14.117123 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 234/459] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:24:14.221210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3869bf0-bb0d-4875-96d2-63f3a20ab84b None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:14.223118 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3869bf0-bb0d-4875-96d2-63f3a20ab84b None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:14.223118 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3869bf0-bb0d-4875-96d2-63f3a20ab84b None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:14.223588 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 226/460] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:14.228259 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d23632-b90e-49a8-afd4-35c44e8ccf45 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:14.228580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d23632-b90e-49a8-afd4-35c44e8ccf45 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:14.228863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d23632-b90e-49a8-afd4-35c44e8ccf45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:14.257965 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-c9d23632-b90e-49a8-afd4-35c44e8ccf45 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:14.259346 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-c9d23632-b90e-49a8-afd4-35c44e8ccf45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:14.267286 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3869bf0-bb0d-4875-96d2-63f3a20ab84b None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:14.402620 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 235/461] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:24:14.434075 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3869bf0-bb0d-4875-96d2-63f3a20ab84b None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:14.471307 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d23632-b90e-49a8-afd4-35c44e8ccf45 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:14.472781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:14.493173 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b818a2-913a-4ebe-b053-b898a0d53d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:14.493816 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b818a2-913a-4ebe-b053-b898a0d53d36 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:14.494098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b818a2-913a-4ebe-b053-b898a0d53d36 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:14.494371 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b818a2-913a-4ebe-b053-b898a0d53d36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:14.515253 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b818a2-913a-4ebe-b053-b898a0d53d36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:14.516745 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b818a2-913a-4ebe-b053-b898a0d53d36 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:14.548148 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-59b818a2-913a-4ebe-b053-b898a0d53d36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:14.548148 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:14.548148 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:14.548148 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:14.548148 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:14.548148 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:14.548148 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:14.548148 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:14.548148 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:14.548148 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:14.548148 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:14.548148 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:14.548148 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:14.548148 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:14.548148 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:14.548148 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:14.549335 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:14.549335 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:14.549335 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:14.549335 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:14.549335 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:14.549335 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:14.549335 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:14.549335 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:14.549335 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:14.549335 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:14.549335 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:14.549335 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:14.549335 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:14.549335 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:14.549335 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:14.549335 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:14.549335 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:14.550378 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:14.550378 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:14.550378 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:14.553682 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 227/462] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:14.559499 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b818a2-913a-4ebe-b053-b898a0d53d36 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:14.616714 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:14.636378 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d23632-b90e-49a8-afd4-35c44e8ccf45 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:14.674187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d76fb1-f484-4bcb-8c2b-941116614fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:14.675194 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d76fb1-f484-4bcb-8c2b-941116614fb5 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:14.675636 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d76fb1-f484-4bcb-8c2b-941116614fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:14.676055 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d76fb1-f484-4bcb-8c2b-941116614fb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:14.687968 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d76fb1-f484-4bcb-8c2b-941116614fb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:14.689850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d76fb1-f484-4bcb-8c2b-941116614fb5 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:14.706505 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-35d76fb1-f484-4bcb-8c2b-941116614fb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:14.706505 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:14.706505 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:14.706505 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:14.706505 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:14.706505 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:14.706505 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:14.706505 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:14.706505 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:14.706505 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:14.706505 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:14.706505 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:14.706505 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:14.706505 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:14.706505 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:14.706505 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:14.707537 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:14.707537 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:14.707537 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:14.707537 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:14.707537 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:14.707537 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:14.707537 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:14.707537 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:14.707537 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:14.707537 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:14.707537 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:14.707537 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:14.707537 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:14.707537 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:14.707537 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:14.707537 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:14.707537 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:14.708369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:14.708369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:14.708369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:14.710214 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 236/463] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:14.711610 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cb19cc-3305-402f-8b8f-25be72bc3cc8 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:14.712051 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cb19cc-3305-402f-8b8f-25be72bc3cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:14.712517 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cb19cc-3305-402f-8b8f-25be72bc3cc8 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:14.714214 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 237/464] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:14.720271 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cb19cc-3305-402f-8b8f-25be72bc3cc8 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:14.730273 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b818a2-913a-4ebe-b053-b898a0d53d36 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:14.769216 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5377540c-d6aa-4582-ac4d-e4a00199580c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:14.770158 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5377540c-d6aa-4582-ac4d-e4a00199580c None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:14.770608 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5377540c-d6aa-4582-ac4d-e4a00199580c None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:14.771041 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5377540c-d6aa-4582-ac4d-e4a00199580c None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:14.771699 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5377540c-d6aa-4582-ac4d-e4a00199580c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:14.772006 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5377540c-d6aa-4582-ac4d-e4a00199580c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:14.773454 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5377540c-d6aa-4582-ac4d-e4a00199580c None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:14.792508 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 228/465] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:24:14.798685 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5377540c-d6aa-4582-ac4d-e4a00199580c None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:14.831766 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cb19cc-3305-402f-8b8f-25be72bc3cc8 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:14.832058 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:14.865428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:14.871643 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8650780a-dce1-4123-9bc9-74277ed2d6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:14.872460 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650780a-dce1-4123-9bc9-74277ed2d6c4 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:14.872749 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650780a-dce1-4123-9bc9-74277ed2d6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:14.873787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650780a-dce1-4123-9bc9-74277ed2d6c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:14.886155 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8650780a-dce1-4123-9bc9-74277ed2d6c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:14.887680 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8650780a-dce1-4123-9bc9-74277ed2d6c4 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:14.903056 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-8650780a-dce1-4123-9bc9-74277ed2d6c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:14.903056 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:14.903056 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:14.903056 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:14.903056 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:14.903056 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:14.903056 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:14.903056 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:14.903056 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:14.903056 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:14.903056 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:14.903056 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:14.903056 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:14.903056 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:14.903056 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:14.903056 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:14.903954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:14.903954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:14.903954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:14.903954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:14.903954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:14.903954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:14.903954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:14.903954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:14.903954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:14.903954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:14.903954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:14.903954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:14.903954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:14.903954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:14.903954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:14.903954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:14.903954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:14.905186 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:14.905186 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:14.905186 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:14.905186 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 238/466] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:24:14.905406 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ef8ff-e603-4f7a-9bc7-57cf6499e351 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:14.905637 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ef8ff-e603-4f7a-9bc7-57cf6499e351 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:14.906131 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ef8ff-e603-4f7a-9bc7-57cf6499e351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:14.912917 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5377540c-d6aa-4582-ac4d-e4a00199580c None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:14.926753 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-8a3ef8ff-e603-4f7a-9bc7-57cf6499e351 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:14.927877 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-8a3ef8ff-e603-4f7a-9bc7-57cf6499e351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:14.953087 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f1966-f45c-4967-96a3-a9f802b58513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:14.953780 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f1966-f45c-4967-96a3-a9f802b58513 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:14.954064 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f1966-f45c-4967-96a3-a9f802b58513 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:14.954340 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f1966-f45c-4967-96a3-a9f802b58513 None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:14.954985 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728f1966-f45c-4967-96a3-a9f802b58513 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:14.955291 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728f1966-f45c-4967-96a3-a9f802b58513 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:14.956705 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728f1966-f45c-4967-96a3-a9f802b58513 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:14.969766 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 229/467] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:14.975114 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f1966-f45c-4967-96a3-a9f802b58513 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:15.004301 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 239/468] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:24:15.005888 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3ef8ff-e603-4f7a-9bc7-57cf6499e351 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:15.107276 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:15.116636 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f1966-f45c-4967-96a3-a9f802b58513 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:15.136696 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3ef8ff-e603-4f7a-9bc7-57cf6499e351 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:15.153706 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:15.153856 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666c4ef2-3971-44cd-8a69-453204986947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:15.154700 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c4ef2-3971-44cd-8a69-453204986947 None admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:15.154974 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c4ef2-3971-44cd-8a69-453204986947 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:15.155260 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c4ef2-3971-44cd-8a69-453204986947 None admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:15.156819 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666c4ef2-3971-44cd-8a69-453204986947 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:15.158350 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666c4ef2-3971-44cd-8a69-453204986947 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:15.178075 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b376ca8-0bfd-41f5-ae73-8b74753b221e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:15.178733 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b376ca8-0bfd-41f5-ae73-8b74753b221e None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:15.179110 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b376ca8-0bfd-41f5-ae73-8b74753b221e None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:15.179396 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b376ca8-0bfd-41f5-ae73-8b74753b221e None admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:15.180047 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b376ca8-0bfd-41f5-ae73-8b74753b221e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:15.180384 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b376ca8-0bfd-41f5-ae73-8b74753b221e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:15.181891 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b376ca8-0bfd-41f5-ae73-8b74753b221e None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:15.194010 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 240/469] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:15.194829 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b376ca8-0bfd-41f5-ae73-8b74753b221e None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:15.284730 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 230/470] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Aug 24 05:24:14 2022] POST /identity/v3/users => generated 246 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:24:15.286393 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da48ef9-83a4-400b-959e-ca56a99b1a87 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:15.286905 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da48ef9-83a4-400b-959e-ca56a99b1a87 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:15.287558 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da48ef9-83a4-400b-959e-ca56a99b1a87 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:15.289232 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 231/471] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:15.290337 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da48ef9-83a4-400b-959e-ca56a99b1a87 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:15.307404 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b376ca8-0bfd-41f5-ae73-8b74753b221e None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:15.348780 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a7abd-d9b6-49bd-b82d-854995d82b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:15.349505 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a7abd-d9b6-49bd-b82d-854995d82b05 None admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:15.349789 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a7abd-d9b6-49bd-b82d-854995d82b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:15.350062 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a7abd-d9b6-49bd-b82d-854995d82b05 None admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:15.350872 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a7abd-d9b6-49bd-b82d-854995d82b05 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:15.352358 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a7abd-d9b6-49bd-b82d-854995d82b05 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:15.420063 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da48ef9-83a4-400b-959e-ca56a99b1a87 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:15.461342 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5416b7-d737-45b1-ad9d-76297e4e23bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:15.461869 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5416b7-d737-45b1-ad9d-76297e4e23bb None admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:15.462215 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5416b7-d737-45b1-ad9d-76297e4e23bb None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:15.462491 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5416b7-d737-45b1-ad9d-76297e4e23bb None admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:15.463137 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5416b7-d737-45b1-ad9d-76297e4e23bb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:15.464654 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5416b7-d737-45b1-ad9d-76297e4e23bb None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:15.476359 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 241/472] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Aug 24 05:24:15 2022] POST /identity/v3/users => generated 248 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:24:15.477266 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a7abd-d9b6-49bd-b82d-854995d82b05 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:15.513740 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:15.589392 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 232/473] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Aug 24 05:24:15 2022] POST /identity/v3/users => generated 248 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:24:15.595458 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670ef85-91e9-4597-8b4a-939e091c5a99 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:15.595458 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670ef85-91e9-4597-8b4a-939e091c5a99 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:15.595458 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670ef85-91e9-4597-8b4a-939e091c5a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:15.606285 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:15.618726 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a7abd-d9b6-49bd-b82d-854995d82b05 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:15.629974 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-c670ef85-91e9-4597-8b4a-939e091c5a99 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:15.631240 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-c670ef85-91e9-4597-8b4a-939e091c5a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:15.688083 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd499ff5-9577-4c26-b5f8-d157d33499de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:15.688860 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd499ff5-9577-4c26-b5f8-d157d33499de None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:15.689149 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd499ff5-9577-4c26-b5f8-d157d33499de None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:15.689430 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd499ff5-9577-4c26-b5f8-d157d33499de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:15.702114 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd499ff5-9577-4c26-b5f8-d157d33499de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:15.704181 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd499ff5-9577-4c26-b5f8-d157d33499de None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:15.713071 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 233/474] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:24:15.729874 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-fd499ff5-9577-4c26-b5f8-d157d33499de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:15.729874 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:15.729874 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:15.729874 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:15.729874 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:15.729874 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:15.729874 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:15.729874 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:15.729874 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:15.729874 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:15.729874 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:15.729874 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:15.729874 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:15.729874 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:15.729874 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:15.729874 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:15.730838 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:15.730838 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:15.730838 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:15.730838 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:15.730838 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:15.730838 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:15.730838 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:15.730838 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:15.730838 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:15.730838 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:15.730838 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:15.730838 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:15.730838 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:15.730838 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:15.730838 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:15.730838 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:15.730838 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:15.731671 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:15.731671 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:15.731671 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:15.731671 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 242/475] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:15.738582 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c670ef85-91e9-4597-8b4a-939e091c5a99 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:15.850066 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c670ef85-91e9-4597-8b4a-939e091c5a99 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:15.886759 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe023bd0-7b9e-43c5-8776-6acd7aed5368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:15.887379 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe023bd0-7b9e-43c5-8776-6acd7aed5368 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:15.887663 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe023bd0-7b9e-43c5-8776-6acd7aed5368 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:15.887938 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe023bd0-7b9e-43c5-8776-6acd7aed5368 None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:15.888627 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe023bd0-7b9e-43c5-8776-6acd7aed5368 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:15.889003 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe023bd0-7b9e-43c5-8776-6acd7aed5368 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:15.891049 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe023bd0-7b9e-43c5-8776-6acd7aed5368 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:15.893055 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:15.904384 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 234/476] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:15.910866 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd499ff5-9577-4c26-b5f8-d157d33499de None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:16.004006 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:16.026716 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd499ff5-9577-4c26-b5f8-d157d33499de None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:16.037012 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe023bd0-7b9e-43c5-8776-6acd7aed5368 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:16.078171 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076021a2-137c-42e3-89fb-8d14aec8c5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:16.078791 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076021a2-137c-42e3-89fb-8d14aec8c5c9 None admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:16.079071 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076021a2-137c-42e3-89fb-8d14aec8c5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:16.079453 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076021a2-137c-42e3-89fb-8d14aec8c5c9 None admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:16.080227 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076021a2-137c-42e3-89fb-8d14aec8c5c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:16.081700 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076021a2-137c-42e3-89fb-8d14aec8c5c9 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:16.159664 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe023bd0-7b9e-43c5-8776-6acd7aed5368 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:16.198154 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb0a0e-37da-4f83-a0da-bc8ed8cf111b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:16.198765 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb0a0e-37da-4f83-a0da-bc8ed8cf111b None admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:16.199047 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb0a0e-37da-4f83-a0da-bc8ed8cf111b None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:16.199322 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb0a0e-37da-4f83-a0da-bc8ed8cf111b None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:16.200065 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edb0a0e-37da-4f83-a0da-bc8ed8cf111b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:16.201937 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edb0a0e-37da-4f83-a0da-bc8ed8cf111b None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:16.206963 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 243/477] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Aug 24 05:24:15 2022] POST /identity/v3/users => generated 249 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:24:16.247716 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.sql.core [None req-8edb0a0e-37da-4f83-a0da-bc8ed8cf111b None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 05:24:16.247716 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 05:24:16.247716 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [parameters: {'id': 'ce7b0f664115421e8c4b8b5646042baf', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 05:24:16.247716 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81671) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 05:24:16.270170 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-8edb0a0e-37da-4f83-a0da-bc8ed8cf111b None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 05:24:16.270170 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:16.270170 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 05:24:16.270170 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 05:24:16.270170 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 05:24:16.270170 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 05:24:16.270170 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 05:24:16.270170 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 05:24:16.270170 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 05:24:16.270170 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application conn.query(q) Aug 24 05:24:16.270170 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 05:24:16.270170 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 05:24:16.270170 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 05:24:16.270170 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application result.read() Aug 24 05:24:16.270170 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 05:24:16.271059 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 05:24:16.271059 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 05:24:16.271059 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 05:24:16.271059 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 05:24:16.271059 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 05:24:16.271059 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 05:24:16.271059 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 05:24:16.271059 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 05:24:16.271059 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:16.271059 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 05:24:16.271059 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:16.271059 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:16.271059 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 05:24:16.271059 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 05:24:16.271059 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 24 05:24:16.271059 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return ref.to_dict() Aug 24 05:24:16.271059 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 05:24:16.271059 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application next(self.gen) Aug 24 05:24:16.272437 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 05:24:16.272437 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application yield resource Aug 24 05:24:16.272437 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 05:24:16.272437 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application next(self.gen) Aug 24 05:24:16.272437 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 05:24:16.272437 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 05:24:16.272437 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 05:24:16.272437 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 05:24:16.272437 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 05:24:16.272437 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise self.value Aug 24 05:24:16.272437 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 05:24:16.272437 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 05:24:16.272437 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 05:24:16.272437 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application session.commit() Aug 24 05:24:16.272437 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 05:24:16.272437 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 05:24:16.272437 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 05:24:16.273504 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 05:24:16.273504 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 05:24:16.273504 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application self.session.flush() Aug 24 05:24:16.273504 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 05:24:16.273504 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 05:24:16.273504 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 05:24:16.273504 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 05:24:16.273504 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 05:24:16.273504 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application compat.raise_( Aug 24 05:24:16.273504 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 05:24:16.273504 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception Aug 24 05:24:16.273504 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 05:24:16.273504 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 05:24:16.273504 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 05:24:16.273504 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 05:24:16.273504 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 05:24:16.273504 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 05:24:16.274395 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 05:24:16.274395 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 05:24:16.274395 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 05:24:16.274395 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 05:24:16.274395 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 05:24:16.274395 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 05:24:16.274395 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 05:24:16.274395 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 05:24:16.274395 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 05:24:16.274395 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 05:24:16.274395 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 05:24:16.274395 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 05:24:16.274395 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 05:24:16.274395 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 05:24:16.274395 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 05:24:16.274395 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception Aug 24 05:24:16.275306 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 05:24:16.275306 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 05:24:16.275306 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 05:24:16.275306 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 05:24:16.275306 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 05:24:16.275306 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 05:24:16.275306 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 05:24:16.275306 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application conn.query(q) Aug 24 05:24:16.275306 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 05:24:16.275306 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 05:24:16.275306 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 05:24:16.275306 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application result.read() Aug 24 05:24:16.275306 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 05:24:16.275306 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 05:24:16.275306 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 05:24:16.275306 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 05:24:16.277022 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 05:24:16.277022 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 05:24:16.277022 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 05:24:16.277022 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 05:24:16.277022 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 05:24:16.277022 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 05:24:16.277022 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [parameters: {'id': 'ce7b0f664115421e8c4b8b5646042baf', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 05:24:16.277022 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 05:24:16.277022 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:16.277022 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:24:16.277022 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:16.277022 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:16.277022 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:16.277022 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:16.277022 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:16.277022 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:16.279104 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:16.279104 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:16.279104 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:16.279104 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:16.279104 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:16.279104 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:16.279104 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 24 05:24:16.279104 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 24 05:24:16.279104 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:16.279104 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:16.279104 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 24 05:24:16.279104 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 24 05:24:16.279104 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 05:24:16.279104 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 05:24:16.279104 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 05:24:16.279104 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:16.281537 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 235/478] 10.222.0.15 () {60 vars in 1131 bytes} [Wed Aug 24 05:24:16 2022] POST /identity/v3/roles => generated 144 bytes in 235 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:24:16.281676 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076021a2-137c-42e3-89fb-8d14aec8c5c9 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:16.430653 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076021a2-137c-42e3-89fb-8d14aec8c5c9 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:16.467355 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e2907-ee1e-42f3-91e5-83f8fd031ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:16.467999 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e2907-ee1e-42f3-91e5-83f8fd031ea4 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:16.468429 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e2907-ee1e-42f3-91e5-83f8fd031ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:16.468788 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e2907-ee1e-42f3-91e5-83f8fd031ea4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:16.476207 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:16.481404 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2e2907-ee1e-42f3-91e5-83f8fd031ea4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:24:16.486857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-ab2e2907-ee1e-42f3-91e5-83f8fd031ea4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:24:16.486857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:16.486857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:16.486857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:16.486857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:16.486857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:16.486857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:16.486857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:16.486857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:16.486857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:16.486857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:16.486857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:16.486857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:24:16.486857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:24:16.486857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:24:16.486857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:24:16.489185 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:24:16.489185 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:24:16.489185 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:24:16.489185 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:24:16.489185 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:16.489185 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:16.489185 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:16.489185 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:16.489185 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:16.489185 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:16.489185 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:16.489185 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:16.489185 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:16.489185 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:16.489185 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:16.489185 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:16.491194 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:16.491194 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:16.491194 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:16.491194 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:16.491194 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:24:16.491194 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:24:16.491194 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:24:16.491194 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:24:16.491194 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:24:16.491194 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:24:16.491194 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:24:16.491194 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:16.491194 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 244/479] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Aug 24 05:24:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:16.493810 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb0a0e-37da-4f83-a0da-bc8ed8cf111b None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:16.560851 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:16.607763 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb0a0e-37da-4f83-a0da-bc8ed8cf111b None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:16.648309 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b2bde1-389f-4948-9a39-0997a3e20eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:16.648970 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2bde1-389f-4948-9a39-0997a3e20eaa None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:16.649250 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2bde1-389f-4948-9a39-0997a3e20eaa None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:16.649521 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2bde1-389f-4948-9a39-0997a3e20eaa None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:16.650193 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b2bde1-389f-4948-9a39-0997a3e20eaa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:16.650497 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b2bde1-389f-4948-9a39-0997a3e20eaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:16.651976 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b2bde1-389f-4948-9a39-0997a3e20eaa None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:16.661520 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 236/480] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:24:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:17.673698 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c80d1-5fa4-4fe7-b489-4c4bb47744f1 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:17.674207 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c80d1-5fa4-4fe7-b489-4c4bb47744f1 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:17.674282 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c80d1-5fa4-4fe7-b489-4c4bb47744f1 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:17.675883 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 245/481] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:17.698818 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01783adc-67e0-4f0a-9c34-60d9c3596aae None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:17.699349 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01783adc-67e0-4f0a-9c34-60d9c3596aae None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:17.699349 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01783adc-67e0-4f0a-9c34-60d9c3596aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:17.725262 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-01783adc-67e0-4f0a-9c34-60d9c3596aae None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:17.726472 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-01783adc-67e0-4f0a-9c34-60d9c3596aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:17.816233 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b987f62f-7884-49aa-bc87-f20d0c69acdd None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:17.816921 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b987f62f-7884-49aa-bc87-f20d0c69acdd None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:17.817238 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b987f62f-7884-49aa-bc87-f20d0c69acdd None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:17.818642 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 246/482] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:24:17.823821 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c8c69-e01e-4058-b01f-f077a45f00d0 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:17.824176 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c8c69-e01e-4058-b01f-f077a45f00d0 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:17.824545 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c8c69-e01e-4058-b01f-f077a45f00d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:17.834091 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 237/483] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:24:17.847500 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-e62c8c69-e01e-4058-b01f-f077a45f00d0 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:17.849567 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-e62c8c69-e01e-4058-b01f-f077a45f00d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:17.954815 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 247/484] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:24:18.197021 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dbfb22-9ac5-4c0c-beb3-eac684eb68ad None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:18.197650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dbfb22-9ac5-4c0c-beb3-eac684eb68ad None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:18.198140 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dbfb22-9ac5-4c0c-beb3-eac684eb68ad None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:18.199739 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 238/485] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:18.206283 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2d390-1a78-4b26-b7cd-ea13c5b8c08f None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:18.206813 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2d390-1a78-4b26-b7cd-ea13c5b8c08f None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:18.207307 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2d390-1a78-4b26-b7cd-ea13c5b8c08f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:18.223088 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dbfb22-9ac5-4c0c-beb3-eac684eb68ad None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:18.238457 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-3ef2d390-1a78-4b26-b7cd-ea13c5b8c08f None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:18.242069 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-3ef2d390-1a78-4b26-b7cd-ea13c5b8c08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:18.321926 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:18.350078 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 248/486] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:24:18.356420 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2d390-1a78-4b26-b7cd-ea13c5b8c08f None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:18.402275 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dbfb22-9ac5-4c0c-beb3-eac684eb68ad None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:18.458025 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:18.476723 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85173a23-06ae-48a9-a7a0-43927ec32d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:18.477364 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85173a23-06ae-48a9-a7a0-43927ec32d5f None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:18.477647 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85173a23-06ae-48a9-a7a0-43927ec32d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:18.477921 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85173a23-06ae-48a9-a7a0-43927ec32d5f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:18.501004 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85173a23-06ae-48a9-a7a0-43927ec32d5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:24:18.509546 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-85173a23-06ae-48a9-a7a0-43927ec32d5f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:24:18.509546 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:18.509546 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:18.509546 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:18.509546 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:18.509546 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:18.509546 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:18.509546 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:18.509546 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:18.509546 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:18.509546 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:18.509546 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:18.509546 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:24:18.509546 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:24:18.509546 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:24:18.509546 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:24:18.510415 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:24:18.510415 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:24:18.510415 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:24:18.510415 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:24:18.510415 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:18.510415 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:18.510415 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:18.510415 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:18.510415 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:18.510415 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:18.510415 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:18.510415 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:18.510415 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:18.510415 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:18.510415 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:18.510415 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:18.511194 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:18.511194 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:18.511194 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:18.511194 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:18.511194 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:24:18.511194 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:24:18.511194 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:24:18.511194 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:24:18.511194 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:24:18.511194 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:24:18.511194 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:24:18.511194 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:18.516354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 239/487] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Aug 24 05:24:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:24:18.517732 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85173a23-06ae-48a9-a7a0-43927ec32d5f None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:18.540699 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2d390-1a78-4b26-b7cd-ea13c5b8c08f None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:18.598969 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e4a6a-a990-4783-9708-2e2fdf194283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:18.599615 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e4a6a-a990-4783-9708-2e2fdf194283 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:18.599934 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e4a6a-a990-4783-9708-2e2fdf194283 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:18.600244 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e4a6a-a990-4783-9708-2e2fdf194283 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:18.617879 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426e4a6a-a990-4783-9708-2e2fdf194283 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:24:18.623120 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-426e4a6a-a990-4783-9708-2e2fdf194283 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:24:18.623120 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:18.623120 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:18.623120 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:18.623120 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:18.623120 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:18.623120 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:18.623120 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:18.623120 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:18.623120 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:18.623120 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:18.623120 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:18.623120 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:24:18.623120 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:24:18.623120 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:24:18.623120 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:24:18.624017 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:24:18.624017 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:24:18.624017 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:24:18.624017 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:24:18.624017 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:18.624017 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:18.624017 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:18.624017 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:18.624017 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:18.624017 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:18.624017 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:18.624017 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:18.624017 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:18.624017 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:18.624017 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:18.624017 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:18.625653 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:18.625653 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:18.625653 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:18.625653 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:18.625653 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:24:18.625653 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:24:18.625653 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:24:18.625653 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:24:18.625653 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:24:18.625653 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:24:18.625653 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:24:18.625653 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:18.631573 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 249/488] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Aug 24 05:24:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:18.643334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e4a6a-a990-4783-9708-2e2fdf194283 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:18.723538 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85173a23-06ae-48a9-a7a0-43927ec32d5f None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:18.738689 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:18.772710 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e4a6a-a990-4783-9708-2e2fdf194283 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:18.774272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a97991-9e34-4a46-bc7d-12b3bca0e077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:18.774890 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a97991-9e34-4a46-bc7d-12b3bca0e077 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:18.775170 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a97991-9e34-4a46-bc7d-12b3bca0e077 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:18.775440 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a97991-9e34-4a46-bc7d-12b3bca0e077 None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:18.776304 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a97991-9e34-4a46-bc7d-12b3bca0e077 None admin] RBAC: Adding query filter params (name=service) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:18.776684 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a97991-9e34-4a46-bc7d-12b3bca0e077 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:18.779123 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a97991-9e34-4a46-bc7d-12b3bca0e077 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:18.790092 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 240/489] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Aug 24 05:24:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:18.791169 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a97991-9e34-4a46-bc7d-12b3bca0e077 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:18.795803 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:18.819457 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3bccb9-6e61-4541-8bc3-5f32889325bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:18.819457 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3bccb9-6e61-4541-8bc3-5f32889325bf None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:18.819457 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3bccb9-6e61-4541-8bc3-5f32889325bf None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:18.819457 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3bccb9-6e61-4541-8bc3-5f32889325bf None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:18.819849 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3bccb9-6e61-4541-8bc3-5f32889325bf None admin] RBAC: Adding query filter params (name=service) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:18.819849 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3bccb9-6e61-4541-8bc3-5f32889325bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:18.820190 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3bccb9-6e61-4541-8bc3-5f32889325bf None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:18.830717 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 250/490] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Aug 24 05:24:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:24:18.831602 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3bccb9-6e61-4541-8bc3-5f32889325bf None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:18.916736 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a97991-9e34-4a46-bc7d-12b3bca0e077 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:18.955068 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a38bf-a855-457a-908b-9409148b19a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:18.955703 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a38bf-a855-457a-908b-9409148b19a5 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:18.955804 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3bccb9-6e61-4541-8bc3-5f32889325bf None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:18.955990 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a38bf-a855-457a-908b-9409148b19a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:18.956266 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a38bf-a855-457a-908b-9409148b19a5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:18.967954 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197a38bf-a855-457a-908b-9409148b19a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:24:18.974615 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-197a38bf-a855-457a-908b-9409148b19a5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:24:18.974615 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:18.974615 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:18.974615 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:18.974615 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:18.974615 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:18.974615 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:18.974615 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:18.974615 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:18.974615 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:18.974615 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:18.974615 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:18.974615 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:24:18.974615 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:24:18.974615 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:24:18.974615 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:24:18.975521 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:24:18.975521 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:24:18.975521 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:24:18.975521 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:24:18.975521 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:18.975521 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:18.975521 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:18.975521 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:18.975521 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:18.975521 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:18.975521 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:18.975521 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:18.975521 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:18.975521 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:18.975521 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:18.975521 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:18.981312 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:18.981312 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:18.981312 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:18.981312 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:18.981312 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:24:18.981312 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:24:18.981312 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:24:18.981312 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:24:18.981312 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:24:18.981312 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:24:18.981312 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:24:18.981312 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:18.981312 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 241/491] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Aug 24 05:24:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:18.981312 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a38bf-a855-457a-908b-9409148b19a5 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:18.998488 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb37fa6-3ff1-4201-9e55-8850e825a06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:18.999140 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb37fa6-3ff1-4201-9e55-8850e825a06b None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:18.999957 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb37fa6-3ff1-4201-9e55-8850e825a06b None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:19.000389 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb37fa6-3ff1-4201-9e55-8850e825a06b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:19.009921 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:19.011845 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb37fa6-3ff1-4201-9e55-8850e825a06b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:19.013401 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb37fa6-3ff1-4201-9e55-8850e825a06b None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:19.028957 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-bdb37fa6-3ff1-4201-9e55-8850e825a06b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:19.028957 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:19.028957 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:19.028957 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:19.028957 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:19.028957 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:19.028957 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:19.028957 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:19.028957 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:19.028957 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:19.028957 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:19.028957 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:19.028957 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:19.028957 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:19.028957 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:19.028957 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:19.029887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:19.029887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:19.029887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:19.029887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:19.029887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:19.029887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:19.029887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:19.029887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:19.029887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:19.029887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:19.029887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:19.029887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:19.029887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:19.029887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:19.029887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:19.029887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:19.029887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:19.031125 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:19.031125 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:19.031125 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:19.031125 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 251/492] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:19.032038 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6882e5-ffb4-4574-9175-98c06d908fcc None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:19.032252 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6882e5-ffb4-4574-9175-98c06d908fcc None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:19.032493 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6882e5-ffb4-4574-9175-98c06d908fcc None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:19.036525 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 252/493] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:19.036525 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6882e5-ffb4-4574-9175-98c06d908fcc None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:19.071111 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:19.095646 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a38bf-a855-457a-908b-9409148b19a5 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:19.133629 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77e6b4b-cbbe-40ad-8445-dd6a65bea490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:19.134267 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e6b4b-cbbe-40ad-8445-dd6a65bea490 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:19.134601 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e6b4b-cbbe-40ad-8445-dd6a65bea490 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:19.134882 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e6b4b-cbbe-40ad-8445-dd6a65bea490 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:19.146911 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77e6b4b-cbbe-40ad-8445-dd6a65bea490 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:19.148383 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77e6b4b-cbbe-40ad-8445-dd6a65bea490 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:19.149132 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6882e5-ffb4-4574-9175-98c06d908fcc None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:19.165979 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-e77e6b4b-cbbe-40ad-8445-dd6a65bea490 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:19.165979 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:19.165979 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:19.165979 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:19.165979 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:19.165979 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:19.165979 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:19.165979 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:19.165979 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:19.165979 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:19.165979 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:19.165979 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:19.165979 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:19.165979 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:19.165979 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:19.165979 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:19.166942 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:19.166942 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:19.166942 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:19.166942 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:19.166942 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:19.166942 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:19.166942 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:19.166942 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:19.166942 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:19.166942 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:19.166942 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:19.166942 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:19.166942 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:19.166942 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:19.166942 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:19.166942 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:19.166942 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:19.173164 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:19.173164 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:19.173164 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:19.173164 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 242/494] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:19.173164 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77e6b4b-cbbe-40ad-8445-dd6a65bea490 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:19.203545 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb6192-7a8e-4953-8643-008be08b62e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:19.204156 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb6192-7a8e-4953-8643-008be08b62e8 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:19.204460 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb6192-7a8e-4953-8643-008be08b62e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:19.204736 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb6192-7a8e-4953-8643-008be08b62e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:19.205394 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccb6192-7a8e-4953-8643-008be08b62e8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:19.205699 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccb6192-7a8e-4953-8643-008be08b62e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:19.207066 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccb6192-7a8e-4953-8643-008be08b62e8 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:19.215290 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 253/495] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Aug 24 05:24:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:19.216909 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36996f5b-2861-4b7f-a47c-b2297b963df5 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:19.217203 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36996f5b-2861-4b7f-a47c-b2297b963df5 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:19.217483 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36996f5b-2861-4b7f-a47c-b2297b963df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:19.235438 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-36996f5b-2861-4b7f-a47c-b2297b963df5 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:19.236610 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-36996f5b-2861-4b7f-a47c-b2297b963df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:19.260541 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:19.277288 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77e6b4b-cbbe-40ad-8445-dd6a65bea490 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:19.304526 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 254/496] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:24:19.305382 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36996f5b-2861-4b7f-a47c-b2297b963df5 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:19.330260 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:19.331660 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5e0075-f77d-41b8-8496-ef37eeb9ddaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:19.332462 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e0075-f77d-41b8-8496-ef37eeb9ddaa None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:19.332746 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e0075-f77d-41b8-8496-ef37eeb9ddaa None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:19.333389 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e0075-f77d-41b8-8496-ef37eeb9ddaa None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:19.334199 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5e0075-f77d-41b8-8496-ef37eeb9ddaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:19.334514 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5e0075-f77d-41b8-8496-ef37eeb9ddaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:19.335902 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5e0075-f77d-41b8-8496-ef37eeb9ddaa None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:19.347744 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 243/497] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:19.348992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71961aa5-049d-4e58-b8e1-0f56f095652c None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:19.349368 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71961aa5-049d-4e58-b8e1-0f56f095652c None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:19.349651 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71961aa5-049d-4e58-b8e1-0f56f095652c None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:19.355300 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 244/498] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:19.356094 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71961aa5-049d-4e58-b8e1-0f56f095652c None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:19.422159 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36996f5b-2861-4b7f-a47c-b2297b963df5 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:19.460650 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb83152-1216-4060-9f4a-09fdc8c0f6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:19.461264 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb83152-1216-4060-9f4a-09fdc8c0f6fb None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:19.461547 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb83152-1216-4060-9f4a-09fdc8c0f6fb None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:19.461822 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb83152-1216-4060-9f4a-09fdc8c0f6fb None admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:19.462465 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb83152-1216-4060-9f4a-09fdc8c0f6fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:19.462770 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb83152-1216-4060-9f4a-09fdc8c0f6fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:19.463969 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71961aa5-049d-4e58-b8e1-0f56f095652c None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:19.464150 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb83152-1216-4060-9f4a-09fdc8c0f6fb None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:19.477276 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 255/499] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:19.478132 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb83152-1216-4060-9f4a-09fdc8c0f6fb None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:19.509157 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0594b2-beb1-4dde-b1d8-dde487755404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:19.510022 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0594b2-beb1-4dde-b1d8-dde487755404 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:19.510489 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0594b2-beb1-4dde-b1d8-dde487755404 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:19.510776 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0594b2-beb1-4dde-b1d8-dde487755404 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:19.522515 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0594b2-beb1-4dde-b1d8-dde487755404 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:19.523997 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0594b2-beb1-4dde-b1d8-dde487755404 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:19.527186 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:19.541626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-8c0594b2-beb1-4dde-b1d8-dde487755404 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:19.541626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:19.541626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:19.541626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:19.541626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:19.541626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:19.541626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:19.541626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:19.541626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:19.541626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:19.541626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:19.541626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:19.541626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:19.541626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:19.541626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:19.541626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:19.542758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:19.542758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:19.542758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:19.542758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:19.542758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:19.542758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:19.542758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:19.542758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:19.542758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:19.542758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:19.542758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:19.542758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:19.542758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:19.542758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:19.542758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:19.542758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:19.542758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:19.543724 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:19.543724 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:19.543724 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:19.543724 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 245/500] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:19.544051 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ad4b3-90b4-46f1-b02b-cab30643ef91 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:19.544337 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ad4b3-90b4-46f1-b02b-cab30643ef91 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:19.544646 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ad4b3-90b4-46f1-b02b-cab30643ef91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:19.565064 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-692ad4b3-90b4-46f1-b02b-cab30643ef91 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:19.566263 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-692ad4b3-90b4-46f1-b02b-cab30643ef91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:19.576818 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:19.592946 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb83152-1216-4060-9f4a-09fdc8c0f6fb None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:19.627547 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e8a27-4ac2-4a3d-ae45-71e6aae69021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:19.628183 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e8a27-4ac2-4a3d-ae45-71e6aae69021 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:19.628503 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e8a27-4ac2-4a3d-ae45-71e6aae69021 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:19.628789 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e8a27-4ac2-4a3d-ae45-71e6aae69021 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:19.635762 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 246/501] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:24:19.635994 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9e8a27-4ac2-4a3d-ae45-71e6aae69021 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:19.636632 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ad4b3-90b4-46f1-b02b-cab30643ef91 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:19.637487 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9e8a27-4ac2-4a3d-ae45-71e6aae69021 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:19.651850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-ec9e8a27-4ac2-4a3d-ae45-71e6aae69021 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:24:19.651850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:19.651850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:19.651850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:19.651850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:19.651850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:19.651850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:19.651850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:19.651850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:19.651850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:19.651850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:19.651850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:19.651850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:24:19.651850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:24:19.651850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:24:19.651850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:24:19.652816 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:19.652816 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:19.652816 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:24:19.652816 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:19.652816 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:24:19.652816 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:19.652816 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:19.652816 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:19.652816 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:19.652816 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:19.652816 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:19.652816 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:19.652816 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:19.652816 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:19.652816 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:19.652816 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:19.652816 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:19.652816 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:19.653701 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:19.653701 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:19.653701 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:24:19.653701 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:24:19.653701 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:24:19.653701 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:24:19.653701 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:24:19.653701 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:24:19.653701 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:24:19.653701 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:19.654966 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 256/502] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Aug 24 05:24:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:19.656334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e8a27-4ac2-4a3d-ae45-71e6aae69021 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:19.767340 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ad4b3-90b4-46f1-b02b-cab30643ef91 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:19.782088 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e8a27-4ac2-4a3d-ae45-71e6aae69021 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:19.809805 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:19.822035 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67545d1f-2601-46e3-9dc4-6b6596cbe730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:19.823052 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67545d1f-2601-46e3-9dc4-6b6596cbe730 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:19.823474 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67545d1f-2601-46e3-9dc4-6b6596cbe730 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:19.823998 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67545d1f-2601-46e3-9dc4-6b6596cbe730 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:19.835079 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67545d1f-2601-46e3-9dc4-6b6596cbe730 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:19.836838 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e54e1d-0ac0-4811-b50e-67deb618595d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:19.837744 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67545d1f-2601-46e3-9dc4-6b6596cbe730 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:19.837845 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e54e1d-0ac0-4811-b50e-67deb618595d None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:19.838193 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e54e1d-0ac0-4811-b50e-67deb618595d None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:19.838583 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e54e1d-0ac0-4811-b50e-67deb618595d None admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:19.839627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e54e1d-0ac0-4811-b50e-67deb618595d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:19.840126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e54e1d-0ac0-4811-b50e-67deb618595d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:19.842347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e54e1d-0ac0-4811-b50e-67deb618595d None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:19.862557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-67545d1f-2601-46e3-9dc4-6b6596cbe730 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 05:24:19.862557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:19.862557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:19.862557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:19.862557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:19.862557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:19.862557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:19.862557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:19.862557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:19.862557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:19.862557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:19.862557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:19.862557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:24:19.862557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:24:19.862557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:24:19.862557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:24:19.863499 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:19.863499 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:19.863499 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:24:19.863499 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:19.863499 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:24:19.863499 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:19.863499 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:19.863499 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:19.863499 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:19.863499 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:19.863499 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:19.863499 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:19.863499 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:19.863499 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:19.863499 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:19.863499 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:19.863499 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:19.863499 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:19.869557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:19.869557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:19.869557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:24:19.869557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:24:19.869557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:24:19.869557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:24:19.869557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:24:19.869557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:24:19.869557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 05:24:19.869557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:19.870092 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 257/503] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:19.870163 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 247/504] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:24:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:19.870235 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e54e1d-0ac0-4811-b50e-67deb618595d None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:19.870303 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67545d1f-2601-46e3-9dc4-6b6596cbe730 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:19.875177 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:19.975026 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67545d1f-2601-46e3-9dc4-6b6596cbe730 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:19.975535 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e54e1d-0ac0-4811-b50e-67deb618595d None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:20.009555 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153c126-deee-4865-96ca-0442b3234d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:20.010183 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153c126-deee-4865-96ca-0442b3234d2c None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:20.010463 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153c126-deee-4865-96ca-0442b3234d2c None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:20.010565 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd915344-da7a-4436-ae52-251194987afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:20.010731 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153c126-deee-4865-96ca-0442b3234d2c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:20.011104 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd915344-da7a-4436-ae52-251194987afc None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:20.011390 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd915344-da7a-4436-ae52-251194987afc None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:20.011670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd915344-da7a-4436-ae52-251194987afc None admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:20.012415 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd915344-da7a-4436-ae52-251194987afc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:20.012732 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd915344-da7a-4436-ae52-251194987afc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:20.014163 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd915344-da7a-4436-ae52-251194987afc None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:20.020892 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8153c126-deee-4865-96ca-0442b3234d2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:24:20.026678 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-8153c126-deee-4865-96ca-0442b3234d2c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:24:20.026678 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:20.026678 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:20.026678 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:20.026678 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:20.026678 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:20.026678 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:20.026678 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:20.026678 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:20.026678 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:20.026678 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:20.026678 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:20.026678 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:24:20.026678 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:24:20.026678 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:24:20.026678 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:24:20.027603 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:24:20.027603 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:24:20.027603 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:24:20.027603 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:24:20.027603 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:20.027603 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:20.027603 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:20.027603 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:20.027603 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:20.027603 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:20.027603 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:20.027603 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:20.027603 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:20.027603 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:20.027603 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:20.027603 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:20.028443 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:20.028443 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:20.028443 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:20.028443 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:20.028443 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:24:20.028443 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:24:20.028443 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:24:20.028443 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:24:20.028443 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:24:20.028443 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:24:20.028443 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:24:20.028443 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:20.028443 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 258/505] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Aug 24 05:24:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:20.029110 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153c126-deee-4865-96ca-0442b3234d2c None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:20.059522 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 248/506] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Aug 24 05:24:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:20.059522 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd915344-da7a-4436-ae52-251194987afc None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:20.094265 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:20.101666 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:20.132170 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153c126-deee-4865-96ca-0442b3234d2c None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:20.155970 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd915344-da7a-4436-ae52-251194987afc None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:20.166795 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357e724-3caa-4d25-9a5d-c8f54ae902ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:20.167496 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357e724-3caa-4d25-9a5d-c8f54ae902ad None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:20.167781 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357e724-3caa-4d25-9a5d-c8f54ae902ad None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:20.168056 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357e724-3caa-4d25-9a5d-c8f54ae902ad None admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:20.168829 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9357e724-3caa-4d25-9a5d-c8f54ae902ad None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:20.169140 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9357e724-3caa-4d25-9a5d-c8f54ae902ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:20.170603 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9357e724-3caa-4d25-9a5d-c8f54ae902ad None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:20.192676 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a36661-6009-4321-a35d-4b3d1c620971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:20.193312 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a36661-6009-4321-a35d-4b3d1c620971 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:20.193603 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a36661-6009-4321-a35d-4b3d1c620971 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:20.193864 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a36661-6009-4321-a35d-4b3d1c620971 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:20.201486 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a36661-6009-4321-a35d-4b3d1c620971 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:20.203533 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a36661-6009-4321-a35d-4b3d1c620971 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:20.213373 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 259/507] 10.222.0.15 () {56 vars in 1137 bytes} [Wed Aug 24 05:24:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:20.213373 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357e724-3caa-4d25-9a5d-c8f54ae902ad None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:20.214824 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-b9a36661-6009-4321-a35d-4b3d1c620971 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 05:24:20.214824 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:20.214824 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:20.214824 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:20.214824 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:20.214824 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:20.214824 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:20.214824 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:20.214824 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:20.214824 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:20.214824 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:20.214824 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:20.214824 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:24:20.214824 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:24:20.214824 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:24:20.214824 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:24:20.215853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:20.215853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:20.215853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:24:20.215853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:20.215853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:24:20.215853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:20.215853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:20.215853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:20.215853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:20.215853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:20.215853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:20.215853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:20.215853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:20.215853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:20.215853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:20.215853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:20.215853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:20.215853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:20.217152 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:20.217152 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:20.217152 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:24:20.217152 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:24:20.217152 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:24:20.217152 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:24:20.217152 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:24:20.217152 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:24:20.217152 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 05:24:20.217152 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:20.217152 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 249/508] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:24:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:20.217152 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a36661-6009-4321-a35d-4b3d1c620971 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:20.319580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357e724-3caa-4d25-9a5d-c8f54ae902ad None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:20.320376 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a36661-6009-4321-a35d-4b3d1c620971 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:20.322696 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:20.342804 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:20.355218 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64b2c97-e75f-41de-ae5f-aafca6f586d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:20.355850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b2c97-e75f-41de-ae5f-aafca6f586d4 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:20.356134 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b2c97-e75f-41de-ae5f-aafca6f586d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:20.356371 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b019a-eaef-4674-9794-5b77f0319094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:20.356509 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b2c97-e75f-41de-ae5f-aafca6f586d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:20.357558 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b019a-eaef-4674-9794-5b77f0319094 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:20.357843 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b019a-eaef-4674-9794-5b77f0319094 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:20.358116 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b019a-eaef-4674-9794-5b77f0319094 None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:20.359197 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81b019a-eaef-4674-9794-5b77f0319094 None admin] RBAC: Adding query filter params (name=service) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:20.359510 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81b019a-eaef-4674-9794-5b77f0319094 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:20.360988 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81b019a-eaef-4674-9794-5b77f0319094 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:20.366755 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64b2c97-e75f-41de-ae5f-aafca6f586d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:20.368248 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64b2c97-e75f-41de-ae5f-aafca6f586d4 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:20.369041 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 250/509] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Aug 24 05:24:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:20.369917 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b019a-eaef-4674-9794-5b77f0319094 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:20.382929 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-f64b2c97-e75f-41de-ae5f-aafca6f586d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:20.382929 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:20.382929 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:20.382929 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:20.382929 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:20.382929 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:20.382929 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:20.382929 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:20.382929 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:20.382929 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:20.382929 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:20.382929 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:20.382929 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:20.382929 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:20.382929 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:20.382929 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:20.383999 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:20.383999 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:20.383999 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:20.383999 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:20.383999 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:20.383999 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:20.383999 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:20.383999 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:20.383999 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:20.383999 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:20.383999 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:20.383999 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:20.383999 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:20.383999 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:20.383999 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:20.383999 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:20.383999 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:20.384883 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:20.384883 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:20.384883 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:20.384883 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 260/510] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:20.384883 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64b2c97-e75f-41de-ae5f-aafca6f586d4 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:20.472776 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b019a-eaef-4674-9794-5b77f0319094 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:20.486540 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64b2c97-e75f-41de-ae5f-aafca6f586d4 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:20.508375 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43285727-724b-4fb6-af43-997c93fb8510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:20.509009 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43285727-724b-4fb6-af43-997c93fb8510 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:20.509292 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43285727-724b-4fb6-af43-997c93fb8510 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:20.509577 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43285727-724b-4fb6-af43-997c93fb8510 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:20.520372 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e3d65-a350-41f0-9c41-b01a8ff1774e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:20.521234 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e3d65-a350-41f0-9c41-b01a8ff1774e None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:20.521510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e3d65-a350-41f0-9c41-b01a8ff1774e None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:20.521780 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e3d65-a350-41f0-9c41-b01a8ff1774e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:20.543697 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:20.563392 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:20.578323 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43285727-724b-4fb6-af43-997c93fb8510 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:20.579736 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43285727-724b-4fb6-af43-997c93fb8510 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:20.588419 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4e3d65-a350-41f0-9c41-b01a8ff1774e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:20.589907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4e3d65-a350-41f0-9c41-b01a8ff1774e None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:20.686544 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 251/511] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:24:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:24:20.688343 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43285727-724b-4fb6-af43-997c93fb8510 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:20.700178 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 261/512] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:24:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:24:20.701076 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e3d65-a350-41f0-9c41-b01a8ff1774e None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:20.755030 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:20.775906 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:20.790432 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43285727-724b-4fb6-af43-997c93fb8510 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:20.800023 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e3d65-a350-41f0-9c41-b01a8ff1774e None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:20.825535 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8514da63-beaa-4501-8ace-07a831314071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:20.826150 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8514da63-beaa-4501-8ace-07a831314071 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:20.826432 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8514da63-beaa-4501-8ace-07a831314071 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:20.826707 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8514da63-beaa-4501-8ace-07a831314071 None admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:20.827446 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8514da63-beaa-4501-8ace-07a831314071 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:20.828473 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8514da63-beaa-4501-8ace-07a831314071 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:20.829883 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8514da63-beaa-4501-8ace-07a831314071 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:20.834306 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426027da-423c-4d25-aa0c-a8d9f49acd11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:20.834998 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426027da-423c-4d25-aa0c-a8d9f49acd11 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:20.835279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426027da-423c-4d25-aa0c-a8d9f49acd11 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:20.840777 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426027da-423c-4d25-aa0c-a8d9f49acd11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:20.846034 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426027da-423c-4d25-aa0c-a8d9f49acd11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:20.846799 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 252/513] 10.222.0.15 () {56 vars in 1137 bytes} [Wed Aug 24 05:24:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:20.847467 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426027da-423c-4d25-aa0c-a8d9f49acd11 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:20.847641 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8514da63-beaa-4501-8ace-07a831314071 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:20.862450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-426027da-423c-4d25-aa0c-a8d9f49acd11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:20.862450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:20.862450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:20.862450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:20.862450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:20.862450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:20.862450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:20.862450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:20.862450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:20.862450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:20.862450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:20.862450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:20.862450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:20.862450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:20.862450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:20.862450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:20.863557 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:20.863557 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:20.863557 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:20.863557 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:20.863557 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:20.863557 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:20.863557 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:20.863557 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:20.863557 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:20.863557 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:20.863557 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:20.863557 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:20.863557 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:20.863557 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:20.863557 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:20.863557 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:20.863557 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:20.864587 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:20.864587 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:20.864587 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:20.864587 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 262/514] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:20.864845 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426027da-423c-4d25-aa0c-a8d9f49acd11 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:20.950595 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8514da63-beaa-4501-8ace-07a831314071 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:20.964729 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426027da-423c-4d25-aa0c-a8d9f49acd11 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:20.974548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:20.986178 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7540def4-d4ab-41df-b09c-85f6d8070d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:20.986788 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540def4-d4ab-41df-b09c-85f6d8070d96 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:20.987073 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540def4-d4ab-41df-b09c-85f6d8070d96 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:20.987351 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540def4-d4ab-41df-b09c-85f6d8070d96 None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:20.987997 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7540def4-d4ab-41df-b09c-85f6d8070d96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:20.988308 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7540def4-d4ab-41df-b09c-85f6d8070d96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:20.989917 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7540def4-d4ab-41df-b09c-85f6d8070d96 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:20.991337 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:20.999913 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c14f87-0fa2-415e-8c07-f286bd0dd897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:21.000617 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c14f87-0fa2-415e-8c07-f286bd0dd897 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:21.000906 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 253/515] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:21.001016 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c14f87-0fa2-415e-8c07-f286bd0dd897 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:21.001173 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c14f87-0fa2-415e-8c07-f286bd0dd897 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:21.001706 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7540def4-d4ab-41df-b09c-85f6d8070d96 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:21.012274 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c14f87-0fa2-415e-8c07-f286bd0dd897 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:21.013797 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c14f87-0fa2-415e-8c07-f286bd0dd897 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:21.029048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-58c14f87-0fa2-415e-8c07-f286bd0dd897 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:21.029048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:21.029048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:21.029048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:21.029048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:21.029048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:21.029048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:21.029048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:21.029048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:21.029048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:21.029048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:21.029048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:21.029048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:21.029048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:21.029048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:21.029048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:21.031273 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:21.031273 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:21.031273 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:21.031273 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:21.031273 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:21.031273 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:21.031273 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:21.031273 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:21.031273 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:21.031273 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:21.031273 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:21.031273 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:21.031273 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:21.031273 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:21.031273 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:21.031273 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:21.031273 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:21.032474 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:21.032474 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:21.032474 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:21.032474 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 263/516] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:21.032474 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c14f87-0fa2-415e-8c07-f286bd0dd897 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:21.110398 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7540def4-d4ab-41df-b09c-85f6d8070d96 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:21.133238 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c14f87-0fa2-415e-8c07-f286bd0dd897 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:21.145391 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c940e5d1-3b4a-48c9-b839-94e45902380c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:21.146027 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940e5d1-3b4a-48c9-b839-94e45902380c None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:21.146318 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940e5d1-3b4a-48c9-b839-94e45902380c None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:21.146598 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940e5d1-3b4a-48c9-b839-94e45902380c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:21.157835 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c940e5d1-3b4a-48c9-b839-94e45902380c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:21.159305 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c940e5d1-3b4a-48c9-b839-94e45902380c None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:21.167705 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3625e29-44b3-4b6b-9187-ab8045389409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:21.168308 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3625e29-44b3-4b6b-9187-ab8045389409 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:21.168620 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3625e29-44b3-4b6b-9187-ab8045389409 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:21.168897 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3625e29-44b3-4b6b-9187-ab8045389409 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:21.174324 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-c940e5d1-3b4a-48c9-b839-94e45902380c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:21.174324 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:21.174324 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:21.174324 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:21.174324 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:21.174324 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:21.174324 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:21.174324 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:21.174324 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:21.174324 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:21.174324 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:21.174324 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:21.174324 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:21.174324 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:21.174324 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:21.174324 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:21.175394 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:21.175394 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:21.175394 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:21.175394 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:21.175394 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:21.175394 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:21.175394 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:21.175394 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:21.175394 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:21.175394 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:21.175394 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:21.175394 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:21.175394 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:21.175394 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:21.175394 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:21.175394 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:21.175394 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:21.176508 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:21.176508 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:21.176508 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:21.176508 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 254/517] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:21.176809 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c940e5d1-3b4a-48c9-b839-94e45902380c None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:21.213775 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:21.224079 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:21.245399 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3625e29-44b3-4b6b-9187-ab8045389409 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:21.246919 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3625e29-44b3-4b6b-9187-ab8045389409 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:21.279576 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c940e5d1-3b4a-48c9-b839-94e45902380c None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:21.315558 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216e725-7306-4a8d-b56b-56c48ac3652f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:21.316165 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216e725-7306-4a8d-b56b-56c48ac3652f None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:21.316475 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216e725-7306-4a8d-b56b-56c48ac3652f None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:21.316757 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216e725-7306-4a8d-b56b-56c48ac3652f None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:21.317400 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6216e725-7306-4a8d-b56b-56c48ac3652f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:21.317708 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6216e725-7306-4a8d-b56b-56c48ac3652f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:21.319101 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6216e725-7306-4a8d-b56b-56c48ac3652f None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:21.331277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 255/518] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:21.333188 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216e725-7306-4a8d-b56b-56c48ac3652f None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:21.346618 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 264/519] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:24:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:24:21.347461 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3625e29-44b3-4b6b-9187-ab8045389409 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:21.426794 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:21.432685 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:21.438543 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216e725-7306-4a8d-b56b-56c48ac3652f None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:21.448149 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3625e29-44b3-4b6b-9187-ab8045389409 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:21.472509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e4248-b6b8-4e20-be85-98a35b0b9479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:21.473117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e4248-b6b8-4e20-be85-98a35b0b9479 None admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:21.473398 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e4248-b6b8-4e20-be85-98a35b0b9479 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:21.473678 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e4248-b6b8-4e20-be85-98a35b0b9479 None admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:21.474409 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3e4248-b6b8-4e20-be85-98a35b0b9479 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:21.475835 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3e4248-b6b8-4e20-be85-98a35b0b9479 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:21.484708 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f265da4a-e260-407e-bffc-a696d063c489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:21.485319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265da4a-e260-407e-bffc-a696d063c489 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:21.485589 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265da4a-e260-407e-bffc-a696d063c489 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:21.485866 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265da4a-e260-407e-bffc-a696d063c489 None admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:21.486593 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f265da4a-e260-407e-bffc-a696d063c489 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:21.486896 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f265da4a-e260-407e-bffc-a696d063c489 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:21.488268 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f265da4a-e260-407e-bffc-a696d063c489 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:21.499475 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 265/520] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:21.500263 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f265da4a-e260-407e-bffc-a696d063c489 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:21.596323 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f265da4a-e260-407e-bffc-a696d063c489 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:21.628878 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 256/521] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Aug 24 05:24:21 2022] POST /identity/v3/users => generated 247 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:24:21.629415 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ece699-f3c9-4e5a-a6e1-2493a845fe9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:21.629806 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e4248-b6b8-4e20-be85-98a35b0b9479 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:21.630036 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ece699-f3c9-4e5a-a6e1-2493a845fe9f None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:21.630321 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ece699-f3c9-4e5a-a6e1-2493a845fe9f None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:21.630610 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ece699-f3c9-4e5a-a6e1-2493a845fe9f None admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:21.631260 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ece699-f3c9-4e5a-a6e1-2493a845fe9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:21.631568 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ece699-f3c9-4e5a-a6e1-2493a845fe9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:21.633023 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ece699-f3c9-4e5a-a6e1-2493a845fe9f None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:21.643045 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:21.650084 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 266/522] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:21.653492 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ece699-f3c9-4e5a-a6e1-2493a845fe9f None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:21.757851 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e4248-b6b8-4e20-be85-98a35b0b9479 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:21.771248 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ece699-f3c9-4e5a-a6e1-2493a845fe9f None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:21.802207 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc73478-fe54-4cd8-a5ce-4879cb81824e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:21.802879 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc73478-fe54-4cd8-a5ce-4879cb81824e None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:21.803116 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc73478-fe54-4cd8-a5ce-4879cb81824e None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:21.803387 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc73478-fe54-4cd8-a5ce-4879cb81824e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:21.811076 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc73478-fe54-4cd8-a5ce-4879cb81824e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:21.816532 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad735beb-ddc8-4cc1-b985-9e195c2146d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:21.817113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc73478-fe54-4cd8-a5ce-4879cb81824e None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:21.817209 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad735beb-ddc8-4cc1-b985-9e195c2146d8 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:21.817209 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad735beb-ddc8-4cc1-b985-9e195c2146d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:21.817209 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad735beb-ddc8-4cc1-b985-9e195c2146d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:21.823946 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-dfc73478-fe54-4cd8-a5ce-4879cb81824e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 05:24:21.823946 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:21.823946 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:21.823946 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:21.823946 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:21.823946 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:21.823946 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:21.823946 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:21.823946 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:21.823946 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:21.823946 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:21.823946 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:21.823946 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:24:21.823946 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:24:21.823946 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:24:21.823946 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:24:21.825164 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:21.825164 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:21.825164 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:24:21.825164 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:21.825164 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:24:21.825164 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:21.825164 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:21.825164 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:21.825164 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:21.825164 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:21.825164 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:21.825164 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:21.825164 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:21.825164 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:21.825164 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:21.825164 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:21.825164 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:21.825164 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:21.826256 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:21.826256 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:21.826256 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:24:21.826256 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:24:21.826256 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:24:21.826256 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:24:21.826256 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:24:21.826256 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:24:21.826256 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 05:24:21.826256 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:21.826792 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad735beb-ddc8-4cc1-b985-9e195c2146d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:21.826792 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad735beb-ddc8-4cc1-b985-9e195c2146d8 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:21.826928 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 257/523] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Aug 24 05:24:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:21.830007 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc73478-fe54-4cd8-a5ce-4879cb81824e None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:21.846425 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-ad735beb-ddc8-4cc1-b985-9e195c2146d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:21.846425 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:21.846425 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:21.846425 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:21.846425 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:21.846425 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:21.846425 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:21.846425 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:21.846425 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:21.846425 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:21.846425 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:21.846425 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:21.846425 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:21.846425 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:21.846425 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:21.846425 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:21.847646 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:21.847646 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:21.847646 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:21.847646 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:21.847646 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:21.847646 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:21.847646 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:21.847646 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:21.847646 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:21.847646 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:21.847646 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:21.847646 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:21.847646 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:21.847646 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:21.847646 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:21.847646 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:21.847646 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:21.848684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:21.848684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:21.848684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:21.848684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 267/524] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:21.849002 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:21.849256 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad735beb-ddc8-4cc1-b985-9e195c2146d8 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:21.905313 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:21.945947 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc73478-fe54-4cd8-a5ce-4879cb81824e None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:21.957807 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad735beb-ddc8-4cc1-b985-9e195c2146d8 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:21.985405 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bd8530-5f9a-44f2-ba4a-bd7a1af9ac40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:21.986039 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd8530-5f9a-44f2-ba4a-bd7a1af9ac40 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:21.986317 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd8530-5f9a-44f2-ba4a-bd7a1af9ac40 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:21.986596 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd8530-5f9a-44f2-ba4a-bd7a1af9ac40 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:21.993450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a11d7fb-fbc6-427b-8a54-4b4ddf84b9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:21.993608 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bd8530-5f9a-44f2-ba4a-bd7a1af9ac40 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:21.994262 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11d7fb-fbc6-427b-8a54-4b4ddf84b9d2 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:21.994611 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11d7fb-fbc6-427b-8a54-4b4ddf84b9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:21.994949 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11d7fb-fbc6-427b-8a54-4b4ddf84b9d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:21.995054 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bd8530-5f9a-44f2-ba4a-bd7a1af9ac40 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:22.002550 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a11d7fb-fbc6-427b-8a54-4b4ddf84b9d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:22.004091 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a11d7fb-fbc6-427b-8a54-4b4ddf84b9d2 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:22.008675 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-f0bd8530-5f9a-44f2-ba4a-bd7a1af9ac40 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:22.008675 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:22.008675 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:22.008675 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:22.008675 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:22.008675 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:22.008675 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:22.008675 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:22.008675 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:22.008675 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:22.008675 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:22.008675 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:22.008675 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:24:22.008675 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:24:22.008675 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:24:22.009781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:24:22.009781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:22.009781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:22.009781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:22.009781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:22.009781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:22.009781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:22.009781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:22.009781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:22.009781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:22.009781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:22.009781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:22.009781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:22.009781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:22.009781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:22.009781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:22.009781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:22.010808 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:24:22.010808 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:24:22.010808 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:24:22.010808 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:24:22.010808 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:24:22.010808 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:24:22.010808 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:22.010808 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:22.012709 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 258/525] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:24:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:22.013708 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bd8530-5f9a-44f2-ba4a-bd7a1af9ac40 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:22.018334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-2a11d7fb-fbc6-427b-8a54-4b4ddf84b9d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:22.018334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:22.018334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:22.018334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:22.018334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:22.018334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:22.018334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:22.018334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:22.018334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:22.018334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:22.018334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:22.018334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:22.018334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:24:22.018334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:24:22.018334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:24:22.019369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:24:22.019369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:22.019369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:22.019369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:22.019369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:22.019369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:22.019369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:22.019369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:22.019369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:22.019369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:22.019369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:22.019369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:22.019369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:22.019369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:22.019369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:22.019369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:22.019369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:22.020443 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:24:22.020443 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:24:22.020443 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:24:22.020443 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:24:22.020443 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:24:22.020443 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:24:22.020443 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:22.020443 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:22.024388 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 268/526] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:24:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:22.025288 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a11d7fb-fbc6-427b-8a54-4b4ddf84b9d2 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:22.099235 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:22.129738 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bd8530-5f9a-44f2-ba4a-bd7a1af9ac40 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:22.131909 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a11d7fb-fbc6-427b-8a54-4b4ddf84b9d2 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:22.143041 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:22.168832 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f006e9-b887-40cc-85de-7ed7ea328c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:22.169398 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7da1688-443a-467e-9ba1-508447eb8419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:22.169537 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f006e9-b887-40cc-85de-7ed7ea328c9f None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:22.169708 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f006e9-b887-40cc-85de-7ed7ea328c9f None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:22.170039 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f006e9-b887-40cc-85de-7ed7ea328c9f None admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:22.170149 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da1688-443a-467e-9ba1-508447eb8419 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:22.170316 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da1688-443a-467e-9ba1-508447eb8419 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:22.170590 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da1688-443a-467e-9ba1-508447eb8419 None admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:22.170680 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f006e9-b887-40cc-85de-7ed7ea328c9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:22.170969 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f006e9-b887-40cc-85de-7ed7ea328c9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:22.171913 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7da1688-443a-467e-9ba1-508447eb8419 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:22.172227 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7da1688-443a-467e-9ba1-508447eb8419 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:22.173178 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f006e9-b887-40cc-85de-7ed7ea328c9f None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:22.173678 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7da1688-443a-467e-9ba1-508447eb8419 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:22.188742 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 269/527] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:22.189617 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f006e9-b887-40cc-85de-7ed7ea328c9f None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:22.205837 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 259/528] 10.222.0.15 () {56 vars in 1139 bytes} [Wed Aug 24 05:24:22 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:22.205837 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7da1688-443a-467e-9ba1-508447eb8419 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:22.306780 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f006e9-b887-40cc-85de-7ed7ea328c9f None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:22.321588 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7da1688-443a-467e-9ba1-508447eb8419 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:22.346172 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8577b8-e912-4d21-9b2d-a654b8fa6935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:22.346792 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8577b8-e912-4d21-9b2d-a654b8fa6935 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:22.347074 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8577b8-e912-4d21-9b2d-a654b8fa6935 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:22.347440 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8577b8-e912-4d21-9b2d-a654b8fa6935 None admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:22.348293 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8577b8-e912-4d21-9b2d-a654b8fa6935 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:22.348948 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8577b8-e912-4d21-9b2d-a654b8fa6935 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:22.350962 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8577b8-e912-4d21-9b2d-a654b8fa6935 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:22.353916 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:22.361295 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8159ac36-07c6-4977-a37a-c13c518a7c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:22.362015 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159ac36-07c6-4977-a37a-c13c518a7c62 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:22.362360 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159ac36-07c6-4977-a37a-c13c518a7c62 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:22.362701 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159ac36-07c6-4977-a37a-c13c518a7c62 None admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:22.369484 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8159ac36-07c6-4977-a37a-c13c518a7c62 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:22.369484 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8159ac36-07c6-4977-a37a-c13c518a7c62 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:22.369484 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8159ac36-07c6-4977-a37a-c13c518a7c62 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:22.378233 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 270/529] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Aug 24 05:24:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:22.378233 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8577b8-e912-4d21-9b2d-a654b8fa6935 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:22.385377 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 260/530] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Aug 24 05:24:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:22.386367 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8159ac36-07c6-4977-a37a-c13c518a7c62 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:22.410222 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:22.481926 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8577b8-e912-4d21-9b2d-a654b8fa6935 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:22.503032 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8159ac36-07c6-4977-a37a-c13c518a7c62 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:22.518963 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee03601-2bff-432e-8eea-55b6335abf54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:22.519605 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee03601-2bff-432e-8eea-55b6335abf54 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:22.519890 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee03601-2bff-432e-8eea-55b6335abf54 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:22.520169 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee03601-2bff-432e-8eea-55b6335abf54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:22.531731 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee03601-2bff-432e-8eea-55b6335abf54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:22.533396 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee03601-2bff-432e-8eea-55b6335abf54 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:22.543917 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b1ea69-0582-41dd-bda8-ca5b8900851b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:22.544744 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b1ea69-0582-41dd-bda8-ca5b8900851b None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:22.544965 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-7ee03601-2bff-432e-8eea-55b6335abf54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:22.544965 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:22.544965 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:22.544965 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:22.544965 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:22.544965 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:22.544965 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:22.544965 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:22.544965 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:22.544965 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:22.544965 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:22.544965 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:22.544965 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:24:22.544965 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:24:22.544965 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:24:22.545967 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:24:22.545967 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:22.545967 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:22.545967 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:22.545967 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:22.545967 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:22.545967 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:22.545967 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:22.545967 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:22.545967 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:22.545967 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:22.545967 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:22.545967 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:22.545967 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:22.545967 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:22.545967 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:22.545967 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:22.546913 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:24:22.546913 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:24:22.546913 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:24:22.546913 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:24:22.546913 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:24:22.546913 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:24:22.546913 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:22.546913 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:22.546913 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 271/531] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:24:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:22.547654 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee03601-2bff-432e-8eea-55b6335abf54 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:22.547974 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b1ea69-0582-41dd-bda8-ca5b8900851b None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:22.548414 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b1ea69-0582-41dd-bda8-ca5b8900851b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:22.614611 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:22.626270 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b1ea69-0582-41dd-bda8-ca5b8900851b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:22.627908 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b1ea69-0582-41dd-bda8-ca5b8900851b None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:22.656348 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:22.666886 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee03601-2bff-432e-8eea-55b6335abf54 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:22.705094 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7427fc85-c4cd-4d9a-84d7-c9d628e7e62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:22.705705 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427fc85-c4cd-4d9a-84d7-c9d628e7e62c None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:22.705984 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427fc85-c4cd-4d9a-84d7-c9d628e7e62c None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:22.706254 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427fc85-c4cd-4d9a-84d7-c9d628e7e62c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:22.706980 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7427fc85-c4cd-4d9a-84d7-c9d628e7e62c None admin] RBAC: Adding query filter params (role.id=cb40cd8499f7418ca0da5ea0f1b87ff9, scope.project.id=59592706ae9442e08353158c625e9581, user.id=fbecffb800584c5786813accd061421b) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:22.707289 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7427fc85-c4cd-4d9a-84d7-c9d628e7e62c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:22.708753 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7427fc85-c4cd-4d9a-84d7-c9d628e7e62c None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:22.714955 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 272/532] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:24:22 2022] GET /identity/v3/role_assignments?user.id=fbecffb800584c5786813accd061421b&scope.project.id=59592706ae9442e08353158c625e9581&role.id=cb40cd8499f7418ca0da5ea0f1b87ff9 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:22.716057 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7427fc85-c4cd-4d9a-84d7-c9d628e7e62c None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:22.761327 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 261/533] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:24:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:24:22.762220 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b1ea69-0582-41dd-bda8-ca5b8900851b None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:22.836878 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7427fc85-c4cd-4d9a-84d7-c9d628e7e62c None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:22.871678 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:22.877530 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5bd362-f90f-4597-ac85-ba9fc03ac811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:22.886155 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5bd362-f90f-4597-ac85-ba9fc03ac811 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:22.886155 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5bd362-f90f-4597-ac85-ba9fc03ac811 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:22.886155 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5bd362-f90f-4597-ac85-ba9fc03ac811 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:22.886155 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5bd362-f90f-4597-ac85-ba9fc03ac811 None admin] RBAC: Adding query filter params (role.id=cb40cd8499f7418ca0da5ea0f1b87ff9, scope.project.id=59592706ae9442e08353158c625e9581, user.id=9aee9c524a974a248c3bf363444f584a) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:22.886155 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5bd362-f90f-4597-ac85-ba9fc03ac811 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:22.886155 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5bd362-f90f-4597-ac85-ba9fc03ac811 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:22.886535 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b1ea69-0582-41dd-bda8-ca5b8900851b None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:22.887305 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 273/534] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:24:22 2022] GET /identity/v3/role_assignments?user.id=9aee9c524a974a248c3bf363444f584a&scope.project.id=59592706ae9442e08353158c625e9581&role.id=cb40cd8499f7418ca0da5ea0f1b87ff9 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:22.888176 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5bd362-f90f-4597-ac85-ba9fc03ac811 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:22.916288 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:22.971142 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee672b-99af-4614-ab5f-04df29d0d290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:22.971776 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee672b-99af-4614-ab5f-04df29d0d290 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:22.972065 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee672b-99af-4614-ab5f-04df29d0d290 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:22.972343 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee672b-99af-4614-ab5f-04df29d0d290 None admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:22.973091 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ee672b-99af-4614-ab5f-04df29d0d290 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:22.973403 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ee672b-99af-4614-ab5f-04df29d0d290 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:22.974891 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ee672b-99af-4614-ab5f-04df29d0d290 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:22.999258 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 262/535] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Aug 24 05:24:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:23.005415 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee672b-99af-4614-ab5f-04df29d0d290 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:23.051740 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5bd362-f90f-4597-ac85-ba9fc03ac811 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:23.094243 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a48d7-0aa3-401b-b969-cddc7472000b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:23.094987 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a48d7-0aa3-401b-b969-cddc7472000b None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:23.095249 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a48d7-0aa3-401b-b969-cddc7472000b None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:23.095518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a48d7-0aa3-401b-b969-cddc7472000b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:23.107576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a48d7-0aa3-401b-b969-cddc7472000b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:23.109942 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a48d7-0aa3-401b-b969-cddc7472000b None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:23.125544 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-159a48d7-0aa3-401b-b969-cddc7472000b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:23.125544 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:23.125544 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:23.125544 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:23.125544 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:23.125544 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:23.125544 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:23.125544 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:23.125544 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:23.125544 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:23.125544 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:23.125544 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:23.125544 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:23.125544 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:23.125544 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:23.125544 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:23.126633 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:23.126633 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:23.126633 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:23.126633 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:23.126633 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:23.126633 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:23.126633 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:23.126633 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:23.126633 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:23.126633 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:23.126633 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:23.126633 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:23.126633 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:23.126633 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:23.126633 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:23.126633 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:23.126633 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:23.127457 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:23.127457 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:23.127457 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:23.127457 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 274/536] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:23.136048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a48d7-0aa3-401b-b969-cddc7472000b None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:23.175726 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee672b-99af-4614-ab5f-04df29d0d290 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:23.220549 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27472075-7a87-4a9f-b654-8f55dc89347a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:23.222076 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27472075-7a87-4a9f-b654-8f55dc89347a None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:23.222463 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27472075-7a87-4a9f-b654-8f55dc89347a None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:23.222861 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27472075-7a87-4a9f-b654-8f55dc89347a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:23.223701 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27472075-7a87-4a9f-b654-8f55dc89347a None admin] RBAC: Adding query filter params (role.id=cb40cd8499f7418ca0da5ea0f1b87ff9, scope.project.id=59592706ae9442e08353158c625e9581, user.id=47bfd618ef39498f91fe7731cd40253b) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:23.224118 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27472075-7a87-4a9f-b654-8f55dc89347a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:23.226166 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27472075-7a87-4a9f-b654-8f55dc89347a None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:23.234116 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 263/537] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:24:23 2022] GET /identity/v3/role_assignments?user.id=47bfd618ef39498f91fe7731cd40253b&scope.project.id=59592706ae9442e08353158c625e9581&role.id=cb40cd8499f7418ca0da5ea0f1b87ff9 => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:24:23.265362 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:23.309286 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a48d7-0aa3-401b-b969-cddc7472000b None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:23.361422 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9522539c-1b71-4dec-8d56-8b3f1e5fda05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:23.362049 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522539c-1b71-4dec-8d56-8b3f1e5fda05 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:23.362330 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522539c-1b71-4dec-8d56-8b3f1e5fda05 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:23.362601 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522539c-1b71-4dec-8d56-8b3f1e5fda05 None admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:23.363319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9522539c-1b71-4dec-8d56-8b3f1e5fda05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:23.363625 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9522539c-1b71-4dec-8d56-8b3f1e5fda05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:23.365168 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9522539c-1b71-4dec-8d56-8b3f1e5fda05 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:23.379665 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 275/538] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:23.386446 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27472075-7a87-4a9f-b654-8f55dc89347a None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:23.410463 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:23.512117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27472075-7a87-4a9f-b654-8f55dc89347a None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:23.572053 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0979185-d43c-4ebf-9cca-4591e6a2a1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:23.573085 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0979185-d43c-4ebf-9cca-4591e6a2a1a9 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:23.573590 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0979185-d43c-4ebf-9cca-4591e6a2a1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:23.574063 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0979185-d43c-4ebf-9cca-4591e6a2a1a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:23.587590 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0979185-d43c-4ebf-9cca-4591e6a2a1a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:23.589378 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0979185-d43c-4ebf-9cca-4591e6a2a1a9 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:23.604563 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-a0979185-d43c-4ebf-9cca-4591e6a2a1a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:23.604563 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:23.604563 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:23.604563 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:23.604563 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:23.604563 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:23.604563 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:23.604563 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:23.604563 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:23.604563 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:23.604563 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:23.604563 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:23.604563 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:24:23.604563 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:24:23.604563 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:24:23.605496 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:24:23.605496 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:23.605496 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:23.605496 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:23.605496 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:23.605496 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:23.605496 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:23.605496 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:23.605496 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:23.605496 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:23.605496 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:23.605496 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:23.605496 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:23.605496 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:23.605496 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:23.605496 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:23.605496 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:23.606384 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:24:23.606384 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:24:23.606384 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:24:23.606384 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:24:23.606384 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:24:23.606384 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:24:23.606384 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:23.606384 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:23.608448 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 264/539] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:24:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:23.614957 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9522539c-1b71-4dec-8d56-8b3f1e5fda05 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:23.775881 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9522539c-1b71-4dec-8d56-8b3f1e5fda05 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:23.807033 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:23.813264 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f8ab0-661a-49f0-87fa-0882b5c5d0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:23.813930 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f8ab0-661a-49f0-87fa-0882b5c5d0b8 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:23.814210 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f8ab0-661a-49f0-87fa-0882b5c5d0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:23.814505 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f8ab0-661a-49f0-87fa-0882b5c5d0b8 None admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:23.815193 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2f8ab0-661a-49f0-87fa-0882b5c5d0b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:23.815503 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2f8ab0-661a-49f0-87fa-0882b5c5d0b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:23.816977 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2f8ab0-661a-49f0-87fa-0882b5c5d0b8 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:23.829517 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 276/540] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Aug 24 05:24:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:23.835147 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0979185-d43c-4ebf-9cca-4591e6a2a1a9 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:23.961402 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:23.993120 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0979185-d43c-4ebf-9cca-4591e6a2a1a9 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:24.012967 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b966750-1660-4b43-ad80-14981830682d None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:24.013490 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b966750-1660-4b43-ad80-14981830682d None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:24.013490 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b966750-1660-4b43-ad80-14981830682d None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:24.015768 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 277/541] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:24.032934 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57820ae2-9f24-42e5-81ff-850b2b0ca50e None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:24.033204 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57820ae2-9f24-42e5-81ff-850b2b0ca50e None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:24.033411 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57820ae2-9f24-42e5-81ff-850b2b0ca50e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:24.053874 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-57820ae2-9f24-42e5-81ff-850b2b0ca50e None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:24.055067 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-57820ae2-9f24-42e5-81ff-850b2b0ca50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:24.057643 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c85ff-fe0d-4c45-967c-aa50808c3f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:24.058263 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c85ff-fe0d-4c45-967c-aa50808c3f25 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:24.058550 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c85ff-fe0d-4c45-967c-aa50808c3f25 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:24.058844 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c85ff-fe0d-4c45-967c-aa50808c3f25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:24.059568 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c85ff-fe0d-4c45-967c-aa50808c3f25 None admin] RBAC: Adding query filter params (role.id=cb40cd8499f7418ca0da5ea0f1b87ff9, scope.project.id=59592706ae9442e08353158c625e9581, user.id=b1ecd9b302b64ecf81d7e3d3356ddf7d) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:24.059881 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c85ff-fe0d-4c45-967c-aa50808c3f25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:24.061570 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c85ff-fe0d-4c45-967c-aa50808c3f25 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:24.077712 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 265/542] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:24:23 2022] GET /identity/v3/role_assignments?user.id=b1ecd9b302b64ecf81d7e3d3356ddf7d&scope.project.id=59592706ae9442e08353158c625e9581&role.id=cb40cd8499f7418ca0da5ea0f1b87ff9 => generated 259 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:24.136515 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 278/543] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:24:24.576058 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c85ff-fe0d-4c45-967c-aa50808c3f25 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:24.684974 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c85ff-fe0d-4c45-967c-aa50808c3f25 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:24.739647 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f131cc-9037-4be2-84cc-ccac854f4a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:24.740299 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f131cc-9037-4be2-84cc-ccac854f4a20 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:24.740622 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f131cc-9037-4be2-84cc-ccac854f4a20 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:24.740894 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f131cc-9037-4be2-84cc-ccac854f4a20 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:24.753267 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:24.757262 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f131cc-9037-4be2-84cc-ccac854f4a20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:24:24.769490 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-37f131cc-9037-4be2-84cc-ccac854f4a20 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:24:24.769490 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:24.769490 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:24.769490 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:24.769490 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:24.769490 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:24.769490 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:24.769490 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:24.769490 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:24.769490 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:24.769490 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:24.769490 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:24.769490 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:24:24.769490 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:24:24.769490 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:24:24.769490 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:24:24.770476 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:24:24.770476 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:24:24.770476 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:24:24.770476 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:24:24.770476 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:24.770476 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:24.770476 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:24.770476 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:24.770476 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:24.770476 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:24.770476 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:24.770476 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:24.770476 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:24.770476 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:24.770476 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:24.770476 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:24.771287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:24.771287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:24.771287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:24.771287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:24.771287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:24:24.771287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:24:24.771287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:24:24.771287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:24:24.771287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:24:24.771287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:24:24.771287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:24:24.771287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:24.771287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 266/544] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Aug 24 05:24:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:24.778344 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57820ae2-9f24-42e5-81ff-850b2b0ca50e None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:24.929765 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57820ae2-9f24-42e5-81ff-850b2b0ca50e None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:24.958668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:24.976580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c20fec-8284-4805-ad26-162c4b0e2130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:24.977202 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c20fec-8284-4805-ad26-162c4b0e2130 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:24.977506 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c20fec-8284-4805-ad26-162c4b0e2130 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:24.977816 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c20fec-8284-4805-ad26-162c4b0e2130 None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:24.978491 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c20fec-8284-4805-ad26-162c4b0e2130 None admin] RBAC: Adding query filter params (name=service) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:24.978803 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c20fec-8284-4805-ad26-162c4b0e2130 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:24.980182 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c20fec-8284-4805-ad26-162c4b0e2130 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:24.989663 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 279/545] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Aug 24 05:24:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:24.995810 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f131cc-9037-4be2-84cc-ccac854f4a20 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:25.100315 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05ef6d-be0d-4f4c-94e9-164511f399f2 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:25.100869 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05ef6d-be0d-4f4c-94e9-164511f399f2 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:25.100869 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05ef6d-be0d-4f4c-94e9-164511f399f2 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:25.103443 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 280/546] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:24:25.107458 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1842414-3394-40ec-acba-f568e0549f27 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:25.107744 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1842414-3394-40ec-acba-f568e0549f27 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:25.108026 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1842414-3394-40ec-acba-f568e0549f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:25.118004 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f131cc-9037-4be2-84cc-ccac854f4a20 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:25.126448 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-f1842414-3394-40ec-acba-f568e0549f27 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:25.127655 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-f1842414-3394-40ec-acba-f568e0549f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:25.192447 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db5a94-67fe-4459-baf4-525355723e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:25.192447 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db5a94-67fe-4459-baf4-525355723e31 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:25.192447 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db5a94-67fe-4459-baf4-525355723e31 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:25.192447 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db5a94-67fe-4459-baf4-525355723e31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:25.215215 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db5a94-67fe-4459-baf4-525355723e31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:25.216995 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db5a94-67fe-4459-baf4-525355723e31 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:25.240708 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-b4db5a94-67fe-4459-baf4-525355723e31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:25.240708 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:25.240708 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:25.240708 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:25.240708 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:25.240708 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:25.240708 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:25.240708 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:25.240708 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:25.240708 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:25.240708 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:25.240708 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:25.240708 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:25.240708 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:25.240708 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:25.240708 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:25.241668 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:25.241668 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:25.241668 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:25.241668 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:25.241668 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:25.241668 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:25.241668 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:25.241668 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:25.241668 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:25.241668 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:25.241668 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:25.241668 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:25.241668 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:25.241668 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:25.241668 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:25.241668 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:25.241668 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:25.243433 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:25.243433 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:25.243433 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:25.244842 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 267/547] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:25.247597 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b17c1ba-50de-46da-b5ff-66cc49175696 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:25.247885 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b17c1ba-50de-46da-b5ff-66cc49175696 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:25.248167 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b17c1ba-50de-46da-b5ff-66cc49175696 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:25.250712 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 268/548] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:24:25.251542 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b17c1ba-50de-46da-b5ff-66cc49175696 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:25.276497 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 281/549] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:24:25.277834 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c2685-cdf5-472f-a28d-4b48af3818a1 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:25.278120 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c2685-cdf5-472f-a28d-4b48af3818a1 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:25.278398 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c2685-cdf5-472f-a28d-4b48af3818a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:25.317449 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-630c2685-cdf5-472f-a28d-4b48af3818a1 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:25.320734 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-630c2685-cdf5-472f-a28d-4b48af3818a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:25.335876 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:25.419850 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b17c1ba-50de-46da-b5ff-66cc49175696 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:25.450000 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 282/550] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:24:25.460283 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8789e7-686c-45b3-bb0f-7687db8553e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:25.460990 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8789e7-686c-45b3-bb0f-7687db8553e4 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:25.461281 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8789e7-686c-45b3-bb0f-7687db8553e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:25.461558 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8789e7-686c-45b3-bb0f-7687db8553e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:25.462202 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8789e7-686c-45b3-bb0f-7687db8553e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:25.462568 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8789e7-686c-45b3-bb0f-7687db8553e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:25.464005 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8789e7-686c-45b3-bb0f-7687db8553e4 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:25.477751 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 269/551] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:25.482982 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c2685-cdf5-472f-a28d-4b48af3818a1 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:25.567938 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:25.636788 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c2685-cdf5-472f-a28d-4b48af3818a1 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:25.637103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8789e7-686c-45b3-bb0f-7687db8553e4 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:25.683496 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e16fa91-dcb1-4837-92bd-87cfe145ed97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:25.684188 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16fa91-dcb1-4837-92bd-87cfe145ed97 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:25.684508 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16fa91-dcb1-4837-92bd-87cfe145ed97 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:25.684786 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16fa91-dcb1-4837-92bd-87cfe145ed97 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:25.697472 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e16fa91-dcb1-4837-92bd-87cfe145ed97 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:25.698999 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e16fa91-dcb1-4837-92bd-87cfe145ed97 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:25.711924 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-0e16fa91-dcb1-4837-92bd-87cfe145ed97 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 05:24:25.711924 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:25.711924 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:25.711924 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:25.711924 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:25.711924 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:25.711924 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:25.711924 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:25.711924 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:25.711924 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:25.711924 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:25.711924 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:25.711924 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:24:25.711924 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:24:25.711924 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:24:25.711924 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:24:25.714050 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:25.714050 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:25.714050 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:24:25.714050 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:25.714050 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:24:25.714050 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:25.714050 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:25.714050 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:25.714050 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:25.714050 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:25.714050 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:25.714050 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:25.714050 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:25.714050 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:25.714050 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:25.714050 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:25.714050 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:25.714050 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:25.716204 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:25.716204 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:25.716204 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:24:25.716204 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:24:25.716204 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:24:25.716204 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:24:25.716204 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:24:25.716204 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:24:25.716204 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 05:24:25.716204 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:25.716204 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 283/552] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Aug 24 05:24:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:25.716204 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8cd356-5dc0-4f1e-aeb3-3f094ad56cc8 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:25.716204 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8cd356-5dc0-4f1e-aeb3-3f094ad56cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:25.718342 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8cd356-5dc0-4f1e-aeb3-3f094ad56cc8 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:25.720514 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 284/553] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:25.725780 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2356b5b-747d-40af-867c-d3b0116a7090 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:25.725780 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2356b5b-747d-40af-867c-d3b0116a7090 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:25.725780 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2356b5b-747d-40af-867c-d3b0116a7090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:25.745129 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-f2356b5b-747d-40af-867c-d3b0116a7090 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:25.746338 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-f2356b5b-747d-40af-867c-d3b0116a7090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:25.773072 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8789e7-686c-45b3-bb0f-7687db8553e4 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:25.785973 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:25.812938 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ae371-3354-4c40-9073-34ca90b4dd9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:25.813728 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ae371-3354-4c40-9073-34ca90b4dd9e None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:25.814309 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ae371-3354-4c40-9073-34ca90b4dd9e None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:25.814860 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ae371-3354-4c40-9073-34ca90b4dd9e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:25.818310 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 285/554] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:24:25.825692 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2356b5b-747d-40af-867c-d3b0116a7090 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:25.825833 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3ae371-3354-4c40-9073-34ca90b4dd9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:24:25.830871 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-6a3ae371-3354-4c40-9073-34ca90b4dd9e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:24:25.830871 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:25.830871 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:25.830871 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:25.830871 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:25.830871 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:25.830871 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:25.830871 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:25.830871 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:25.830871 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:25.830871 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:25.830871 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:25.830871 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:24:25.830871 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:24:25.830871 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:24:25.830871 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:24:25.831856 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:24:25.831856 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:24:25.831856 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:24:25.831856 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:24:25.831856 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:25.831856 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:25.831856 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:25.831856 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:25.831856 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:25.831856 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:25.831856 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:25.831856 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:25.831856 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:25.831856 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:25.831856 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:25.831856 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:25.833117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:25.833117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:25.833117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:25.833117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:25.833117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:24:25.833117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:24:25.833117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:24:25.833117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:24:25.833117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:24:25.833117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:24:25.833117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:24:25.833117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:25.836864 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 270/555] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Aug 24 05:24:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:24:25.837817 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ae371-3354-4c40-9073-34ca90b4dd9e None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:25.870745 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:25.962654 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2356b5b-747d-40af-867c-d3b0116a7090 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:25.963352 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ae371-3354-4c40-9073-34ca90b4dd9e None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:25.999297 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea1b735-628b-4004-9c8a-30243dd1ce35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:25.999971 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1b735-628b-4004-9c8a-30243dd1ce35 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:26.000315 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1b735-628b-4004-9c8a-30243dd1ce35 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:26.000626 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1b735-628b-4004-9c8a-30243dd1ce35 None admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:26.001365 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea1b735-628b-4004-9c8a-30243dd1ce35 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:26.001675 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea1b735-628b-4004-9c8a-30243dd1ce35 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:26.003105 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea1b735-628b-4004-9c8a-30243dd1ce35 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:26.011694 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88987ca0-0a0e-4599-832c-f01a054c6e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:26.013106 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88987ca0-0a0e-4599-832c-f01a054c6e8c None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:26.014472 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88987ca0-0a0e-4599-832c-f01a054c6e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:26.015117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88987ca0-0a0e-4599-832c-f01a054c6e8c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:26.027699 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 286/556] 10.222.0.15 () {56 vars in 1135 bytes} [Wed Aug 24 05:24:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:26.027699 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea1b735-628b-4004-9c8a-30243dd1ce35 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:26.035881 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88987ca0-0a0e-4599-832c-f01a054c6e8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:24:26.044594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-88987ca0-0a0e-4599-832c-f01a054c6e8c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:24:26.044594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:26.044594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:26.044594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:26.044594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:26.044594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:26.044594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:26.044594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:26.044594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:26.044594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:26.044594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:26.044594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:26.044594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:24:26.044594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:24:26.044594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:24:26.044594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:24:26.045681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:24:26.045681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:24:26.045681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:24:26.045681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:24:26.045681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:26.045681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:26.045681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:26.045681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:26.045681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:26.045681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:26.045681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:26.045681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:26.045681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:26.045681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:26.045681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:26.045681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:26.047339 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:26.047339 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:26.047339 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:26.047339 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:26.047339 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:24:26.047339 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:24:26.047339 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:24:26.047339 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:24:26.047339 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:24:26.047339 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:24:26.047339 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:24:26.047339 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:26.055288 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 271/557] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Aug 24 05:24:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:26.056747 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88987ca0-0a0e-4599-832c-f01a054c6e8c None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:26.100332 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:26.139102 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:26.151122 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea1b735-628b-4004-9c8a-30243dd1ce35 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:26.174852 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88987ca0-0a0e-4599-832c-f01a054c6e8c None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:26.196771 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775b4592-8eb1-4be5-94fb-e54f92088562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:26.197476 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b4592-8eb1-4be5-94fb-e54f92088562 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:26.197765 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b4592-8eb1-4be5-94fb-e54f92088562 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:26.198044 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b4592-8eb1-4be5-94fb-e54f92088562 None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:26.198714 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775b4592-8eb1-4be5-94fb-e54f92088562 None admin] RBAC: Adding query filter params (name=service) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:26.199020 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775b4592-8eb1-4be5-94fb-e54f92088562 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:26.200386 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775b4592-8eb1-4be5-94fb-e54f92088562 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:26.209615 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 287/558] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Aug 24 05:24:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:26.210265 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775b4592-8eb1-4be5-94fb-e54f92088562 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:26.223294 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea5d886-3a50-481d-9903-ee5045ed68e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:26.223828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea5d886-3a50-481d-9903-ee5045ed68e1 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:26.224114 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea5d886-3a50-481d-9903-ee5045ed68e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:26.224390 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea5d886-3a50-481d-9903-ee5045ed68e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:26.295873 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea5d886-3a50-481d-9903-ee5045ed68e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:26.297411 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea5d886-3a50-481d-9903-ee5045ed68e1 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:26.326434 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:26.330729 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775b4592-8eb1-4be5-94fb-e54f92088562 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:26.366430 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba97c63-305f-4831-848f-f3dc459f7e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:26.367048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba97c63-305f-4831-848f-f3dc459f7e7b None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:26.367326 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba97c63-305f-4831-848f-f3dc459f7e7b None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:26.367828 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba97c63-305f-4831-848f-f3dc459f7e7b None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:26.368245 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba97c63-305f-4831-848f-f3dc459f7e7b None admin] RBAC: Adding query filter params (name=service) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:26.368580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba97c63-305f-4831-848f-f3dc459f7e7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:26.369942 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba97c63-305f-4831-848f-f3dc459f7e7b None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:26.382930 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 288/559] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Aug 24 05:24:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:26.383845 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba97c63-305f-4831-848f-f3dc459f7e7b None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:26.400674 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:26.401972 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 272/560] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:24:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:24:26.402862 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea5d886-3a50-481d-9903-ee5045ed68e1 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:26.495183 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba97c63-305f-4831-848f-f3dc459f7e7b None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:26.510002 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea5d886-3a50-481d-9903-ee5045ed68e1 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:26.530117 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb8712-7ef9-4068-b9ac-de112de0683a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:26.531019 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb8712-7ef9-4068-b9ac-de112de0683a None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:26.531301 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb8712-7ef9-4068-b9ac-de112de0683a None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:26.531573 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb8712-7ef9-4068-b9ac-de112de0683a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:26.541464 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:26.542163 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb8712-7ef9-4068-b9ac-de112de0683a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:24:26.546848 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028e506d-3bee-4673-a611-023a12e88135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:26.547393 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-5cfb8712-7ef9-4068-b9ac-de112de0683a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:24:26.547393 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:26.547393 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:26.547393 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:26.547393 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:26.547393 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:26.547393 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:26.547393 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:26.547393 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:26.547393 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:26.547393 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:26.547393 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:26.547393 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:24:26.547393 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:24:26.547393 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:24:26.547393 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:24:26.548521 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:24:26.548521 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:24:26.548521 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:24:26.548521 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:24:26.548521 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:26.548521 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:26.548521 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:26.548521 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:26.548521 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:26.548521 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:26.548521 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:26.548521 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:26.548521 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:26.548521 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:26.548521 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:26.548521 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:26.549672 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:26.549672 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:26.549672 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:26.549672 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:26.549672 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:24:26.549672 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:24:26.549672 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:24:26.549672 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:24:26.549672 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:24:26.549672 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:24:26.549672 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:24:26.549672 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:26.550409 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e506d-3bee-4673-a611-023a12e88135 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:26.550409 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e506d-3bee-4673-a611-023a12e88135 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:26.550409 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e506d-3bee-4673-a611-023a12e88135 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:26.550586 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 289/561] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Aug 24 05:24:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:26.550586 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb8712-7ef9-4068-b9ac-de112de0683a None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:26.560901 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028e506d-3bee-4673-a611-023a12e88135 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:26.562879 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028e506d-3bee-4673-a611-023a12e88135 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:26.576650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-028e506d-3bee-4673-a611-023a12e88135 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:26.576650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:26.576650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:26.576650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:26.576650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:26.576650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:26.576650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:26.576650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:26.576650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:26.576650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:26.576650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:26.576650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:26.576650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:26.576650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:26.576650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:26.576650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:26.578400 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:26.578400 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:26.578400 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:26.578400 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:26.578400 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:26.578400 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:26.578400 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:26.578400 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:26.578400 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:26.578400 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:26.578400 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:26.578400 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:26.578400 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:26.578400 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:26.578400 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:26.578400 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:26.578400 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:26.579274 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:26.579274 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:26.579274 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:26.579274 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 273/562] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:26.579274 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028e506d-3bee-4673-a611-023a12e88135 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:26.634674 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:26.660524 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb8712-7ef9-4068-b9ac-de112de0683a None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:26.680551 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028e506d-3bee-4673-a611-023a12e88135 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:26.697035 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40024dd-2d70-4647-9c0d-ca66f7dce26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:26.697725 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40024dd-2d70-4647-9c0d-ca66f7dce26e None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:26.698028 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40024dd-2d70-4647-9c0d-ca66f7dce26e None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:26.698299 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40024dd-2d70-4647-9c0d-ca66f7dce26e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:26.714213 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40024dd-2d70-4647-9c0d-ca66f7dce26e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:26.715497 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40024dd-2d70-4647-9c0d-ca66f7dce26e None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:26.720539 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b709bc-c359-412c-b13f-b866e47542f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:26.721161 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b709bc-c359-412c-b13f-b866e47542f4 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:26.721439 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b709bc-c359-412c-b13f-b866e47542f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:26.721725 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b709bc-c359-412c-b13f-b866e47542f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:26.731619 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-b40024dd-2d70-4647-9c0d-ca66f7dce26e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:26.731619 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:26.731619 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:26.731619 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:26.731619 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:26.731619 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:26.731619 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:26.731619 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:26.731619 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:26.731619 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:26.731619 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:26.731619 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:26.731619 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:26.731619 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:26.731619 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:26.731619 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:26.732584 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:26.732584 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:26.732584 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:26.732584 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:26.732584 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:26.732584 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:26.732584 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:26.732584 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:26.732584 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:26.732584 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:26.732584 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:26.732584 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:26.732584 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:26.732584 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:26.732584 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:26.732584 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:26.732584 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:26.733491 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:26.733491 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:26.733491 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:26.734235 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b709bc-c359-412c-b13f-b866e47542f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:26.734306 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 290/563] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:26.734985 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b709bc-c359-412c-b13f-b866e47542f4 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:26.735246 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8fe60-bd36-4f2e-898e-597ce50ff1d7 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:26.735587 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8fe60-bd36-4f2e-898e-597ce50ff1d7 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:26.735859 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8fe60-bd36-4f2e-898e-597ce50ff1d7 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:26.737124 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 291/564] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:26.737822 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd8fe60-bd36-4f2e-898e-597ce50ff1d7 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:26.752257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-74b709bc-c359-412c-b13f-b866e47542f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:26.752257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:26.752257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:26.752257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:26.752257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:26.752257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:26.752257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:26.752257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:26.752257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:26.752257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:26.752257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:26.752257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:26.752257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:26.752257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:26.752257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:26.752257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:26.753561 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:26.753561 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:26.753561 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:26.753561 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:26.753561 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:26.753561 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:26.753561 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:26.753561 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:26.753561 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:26.753561 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:26.753561 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:26.753561 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:26.753561 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:26.753561 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:26.753561 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:26.753561 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:26.753561 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:26.754903 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:26.754903 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:26.754903 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:26.756668 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 274/565] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:26.758149 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b709bc-c359-412c-b13f-b866e47542f4 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:26.793874 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:26.847615 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd8fe60-bd36-4f2e-898e-597ce50ff1d7 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:26.860149 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b709bc-c359-412c-b13f-b866e47542f4 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:26.873884 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:26.882300 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4704f3e1-6507-4d04-804d-8fdfd9b7d13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:26.884351 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704f3e1-6507-4d04-804d-8fdfd9b7d13b None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:26.884351 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704f3e1-6507-4d04-804d-8fdfd9b7d13b None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:26.884351 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704f3e1-6507-4d04-804d-8fdfd9b7d13b None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:26.884351 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4704f3e1-6507-4d04-804d-8fdfd9b7d13b None admin] RBAC: Adding query filter params (name=service) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:26.884351 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4704f3e1-6507-4d04-804d-8fdfd9b7d13b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:26.885741 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4704f3e1-6507-4d04-804d-8fdfd9b7d13b None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:26.894086 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 292/566] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Aug 24 05:24:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:26.894891 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4704f3e1-6507-4d04-804d-8fdfd9b7d13b None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:26.895071 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2570cc-fede-4459-a458-676c5eac249f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:26.895765 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2570cc-fede-4459-a458-676c5eac249f None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:26.896050 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2570cc-fede-4459-a458-676c5eac249f None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:26.896328 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2570cc-fede-4459-a458-676c5eac249f None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:26.897006 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2570cc-fede-4459-a458-676c5eac249f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:26.897316 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2570cc-fede-4459-a458-676c5eac249f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:26.898782 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2570cc-fede-4459-a458-676c5eac249f None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:26.911134 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 275/567] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:26.912497 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941319fd-e99e-4127-bff8-330bc4d86695 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:26.912893 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941319fd-e99e-4127-bff8-330bc4d86695 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:26.913381 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941319fd-e99e-4127-bff8-330bc4d86695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:26.930459 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-941319fd-e99e-4127-bff8-330bc4d86695 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:26.931712 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-941319fd-e99e-4127-bff8-330bc4d86695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:26.999151 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4704f3e1-6507-4d04-804d-8fdfd9b7d13b None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:27.000975 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 276/568] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:24:27.001818 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941319fd-e99e-4127-bff8-330bc4d86695 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:27.029677 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:27.052106 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41e25fc-bf61-425d-b6bf-bbf12f524428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:27.052764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e25fc-bf61-425d-b6bf-bbf12f524428 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:27.053084 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e25fc-bf61-425d-b6bf-bbf12f524428 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:27.053406 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e25fc-bf61-425d-b6bf-bbf12f524428 None admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:27.054049 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41e25fc-bf61-425d-b6bf-bbf12f524428 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:27.054358 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41e25fc-bf61-425d-b6bf-bbf12f524428 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:27.055729 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41e25fc-bf61-425d-b6bf-bbf12f524428 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:27.068146 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 293/569] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:27.069349 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41e25fc-bf61-425d-b6bf-bbf12f524428 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:27.112265 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941319fd-e99e-4127-bff8-330bc4d86695 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:27.121236 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:27.148106 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e00840a-a11f-44c9-80ef-c0b45c643edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:27.148778 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00840a-a11f-44c9-80ef-c0b45c643edb None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:27.149062 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00840a-a11f-44c9-80ef-c0b45c643edb None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:27.149335 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00840a-a11f-44c9-80ef-c0b45c643edb None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:27.149975 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e00840a-a11f-44c9-80ef-c0b45c643edb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:27.150283 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e00840a-a11f-44c9-80ef-c0b45c643edb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:27.151650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e00840a-a11f-44c9-80ef-c0b45c643edb None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:27.167974 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 277/570] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:27.168745 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e00840a-a11f-44c9-80ef-c0b45c643edb None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:27.178749 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41e25fc-bf61-425d-b6bf-bbf12f524428 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:27.222679 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e4b7ea-2686-468c-8f5f-91c43ea470f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:27.223345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e4b7ea-2686-468c-8f5f-91c43ea470f5 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:27.223583 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e4b7ea-2686-468c-8f5f-91c43ea470f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:27.223912 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e4b7ea-2686-468c-8f5f-91c43ea470f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:27.235575 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e4b7ea-2686-468c-8f5f-91c43ea470f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:27.237072 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e4b7ea-2686-468c-8f5f-91c43ea470f5 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:27.252031 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-15e4b7ea-2686-468c-8f5f-91c43ea470f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:27.252031 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:27.252031 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:27.252031 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:27.252031 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:27.252031 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:27.252031 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:27.252031 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:27.252031 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:27.252031 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:27.252031 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:27.252031 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:27.252031 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:27.252031 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:27.252031 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:27.252031 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:27.253068 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:27.253068 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:27.253068 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:27.253068 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:27.253068 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:27.253068 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:27.253068 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:27.253068 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:27.253068 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:27.253068 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:27.253068 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:27.253068 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:27.253068 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:27.253068 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:27.253068 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:27.253068 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:27.253068 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:27.253934 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:27.253934 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:27.253934 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:27.257922 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 294/571] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:27.258674 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e4b7ea-2686-468c-8f5f-91c43ea470f5 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:27.263016 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:27.284785 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e00840a-a11f-44c9-80ef-c0b45c643edb None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:27.320646 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34760a0e-6c94-43de-8e6b-1c86c5c6cbcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:27.321340 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34760a0e-6c94-43de-8e6b-1c86c5c6cbcf None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:27.321620 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34760a0e-6c94-43de-8e6b-1c86c5c6cbcf None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:27.321897 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34760a0e-6c94-43de-8e6b-1c86c5c6cbcf None admin] PATH_INFO: `/v3/users/nova` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:27.329975 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34760a0e-6c94-43de-8e6b-1c86c5c6cbcf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:27.331455 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34760a0e-6c94-43de-8e6b-1c86c5c6cbcf None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:27.343270 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-34760a0e-6c94-43de-8e6b-1c86c5c6cbcf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:24:27.343270 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:27.343270 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:27.343270 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:27.343270 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:27.343270 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:27.343270 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:27.343270 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:27.343270 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:27.343270 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:27.343270 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:27.343270 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:27.343270 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:24:27.343270 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:24:27.343270 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:24:27.343270 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:24:27.344460 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:27.344460 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:27.344460 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:24:27.344460 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:27.344460 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:24:27.344460 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:27.344460 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:27.344460 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:27.344460 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:27.344460 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:27.344460 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:27.344460 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:27.344460 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:27.344460 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:27.344460 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:27.344460 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:27.344460 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:27.344460 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:27.345456 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:27.345456 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:27.345456 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:24:27.345456 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:24:27.345456 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:24:27.345456 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:24:27.345456 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:24:27.345456 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:24:27.345456 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:24:27.345456 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:27.345456 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 278/572] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Aug 24 05:24:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:27.346057 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34760a0e-6c94-43de-8e6b-1c86c5c6cbcf None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:27.365457 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:27.369681 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e4b7ea-2686-468c-8f5f-91c43ea470f5 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:27.408776 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35385004-2a70-4eaa-82f4-5c21d3249c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:27.409415 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35385004-2a70-4eaa-82f4-5c21d3249c0c None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:27.409692 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35385004-2a70-4eaa-82f4-5c21d3249c0c None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:27.409966 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35385004-2a70-4eaa-82f4-5c21d3249c0c None admin] PATH_INFO: `/v3/users/glance` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:27.418375 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35385004-2a70-4eaa-82f4-5c21d3249c0c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:27.419828 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35385004-2a70-4eaa-82f4-5c21d3249c0c None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:27.431848 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-35385004-2a70-4eaa-82f4-5c21d3249c0c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 05:24:27.431848 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:27.431848 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:27.431848 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:27.431848 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:27.431848 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:27.431848 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:27.431848 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:27.431848 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:27.431848 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:27.431848 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:27.431848 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:27.431848 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:24:27.431848 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:24:27.431848 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:24:27.431848 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:24:27.432998 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:27.432998 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:27.432998 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:24:27.432998 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:27.432998 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:24:27.432998 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:27.432998 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:27.432998 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:27.432998 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:27.432998 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:27.432998 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:27.432998 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:27.432998 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:27.432998 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:27.432998 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:27.432998 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:27.432998 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:27.432998 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:27.433926 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:27.433926 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:27.433926 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:24:27.433926 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:24:27.433926 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:24:27.433926 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:24:27.433926 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:24:27.433926 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:24:27.433926 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 05:24:27.433926 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:27.433926 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 295/573] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:24:27 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:27.433926 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35385004-2a70-4eaa-82f4-5c21d3249c0c None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:27.461926 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34760a0e-6c94-43de-8e6b-1c86c5c6cbcf None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:27.496278 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516bcc0c-9d9d-4f1d-9b19-9ec45c978980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:27.496927 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516bcc0c-9d9d-4f1d-9b19-9ec45c978980 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:27.497242 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516bcc0c-9d9d-4f1d-9b19-9ec45c978980 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:27.497516 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516bcc0c-9d9d-4f1d-9b19-9ec45c978980 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:27.504788 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516bcc0c-9d9d-4f1d-9b19-9ec45c978980 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:27.506242 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516bcc0c-9d9d-4f1d-9b19-9ec45c978980 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:27.518141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-516bcc0c-9d9d-4f1d-9b19-9ec45c978980 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:27.518141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:27.518141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:27.518141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:27.518141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:27.518141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:27.518141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:27.518141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:27.518141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:27.518141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:27.518141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:27.518141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:27.518141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:24:27.518141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:24:27.518141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:24:27.519213 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:24:27.519213 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:27.519213 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:27.519213 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:27.519213 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:27.519213 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:27.519213 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:27.519213 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:27.519213 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:27.519213 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:27.519213 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:27.519213 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:27.519213 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:27.519213 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:27.519213 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:27.519213 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:27.519213 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:27.520149 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:24:27.520149 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:24:27.520149 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:24:27.520149 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:24:27.520149 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:24:27.520149 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:24:27.520149 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:27.520149 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:27.520149 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 279/574] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:24:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:27.520149 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516bcc0c-9d9d-4f1d-9b19-9ec45c978980 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:27.524561 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:27.539979 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35385004-2a70-4eaa-82f4-5c21d3249c0c None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:27.574666 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636b9237-791d-4ea9-9d6c-ffb5d2ab4a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:27.575265 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b9237-791d-4ea9-9d6c-ffb5d2ab4a18 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:27.575549 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b9237-791d-4ea9-9d6c-ffb5d2ab4a18 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:27.575823 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b9237-791d-4ea9-9d6c-ffb5d2ab4a18 None admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:27.576498 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636b9237-791d-4ea9-9d6c-ffb5d2ab4a18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:27.576815 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636b9237-791d-4ea9-9d6c-ffb5d2ab4a18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:27.579292 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636b9237-791d-4ea9-9d6c-ffb5d2ab4a18 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:27.590772 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 296/575] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:27.591584 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636b9237-791d-4ea9-9d6c-ffb5d2ab4a18 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:27.614863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:27.622490 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516bcc0c-9d9d-4f1d-9b19-9ec45c978980 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:27.658811 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f6209-4ac0-4174-ab64-38b565abe9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:27.659454 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f6209-4ac0-4174-ab64-38b565abe9fc None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:27.659734 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f6209-4ac0-4174-ab64-38b565abe9fc None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:27.660014 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f6209-4ac0-4174-ab64-38b565abe9fc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:27.670782 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581f6209-4ac0-4174-ab64-38b565abe9fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:24:27.677276 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-581f6209-4ac0-4174-ab64-38b565abe9fc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:24:27.677276 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:27.677276 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:27.677276 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:27.677276 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:27.677276 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:27.677276 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:27.677276 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:27.677276 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:27.677276 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:27.677276 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:27.677276 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:27.677276 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:24:27.677276 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:24:27.677276 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:24:27.677276 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:24:27.678340 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:24:27.678340 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:24:27.678340 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:24:27.678340 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:24:27.678340 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:27.678340 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:27.678340 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:27.678340 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:27.678340 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:27.678340 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:27.678340 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:27.678340 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:27.678340 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:27.678340 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:27.678340 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:27.678340 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:27.679346 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:27.679346 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:27.679346 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:27.679346 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:27.679346 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:24:27.679346 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:24:27.679346 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:24:27.679346 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:24:27.679346 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:24:27.679346 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:24:27.679346 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:24:27.679346 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:27.679346 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 280/576] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Aug 24 05:24:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:27.679346 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f6209-4ac0-4174-ab64-38b565abe9fc None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:27.698725 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636b9237-791d-4ea9-9d6c-ffb5d2ab4a18 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:27.732391 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2462d-bb47-41c4-9616-bc3753a70d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:27.733021 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2462d-bb47-41c4-9616-bc3753a70d11 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:27.733306 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2462d-bb47-41c4-9616-bc3753a70d11 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:27.734110 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2462d-bb47-41c4-9616-bc3753a70d11 None admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:27.735111 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a2462d-bb47-41c4-9616-bc3753a70d11 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:27.735421 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a2462d-bb47-41c4-9616-bc3753a70d11 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:27.736880 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a2462d-bb47-41c4-9616-bc3753a70d11 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:27.744878 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:27.754693 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 297/577] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Aug 24 05:24:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:27.755416 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2462d-bb47-41c4-9616-bc3753a70d11 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:27.785028 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f6209-4ac0-4174-ab64-38b565abe9fc None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:27.819289 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7446d58e-890d-4968-be2c-72c6d6a408b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:27.819815 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446d58e-890d-4968-be2c-72c6d6a408b7 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:27.820099 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446d58e-890d-4968-be2c-72c6d6a408b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:27.820376 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446d58e-890d-4968-be2c-72c6d6a408b7 None admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:27.821132 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7446d58e-890d-4968-be2c-72c6d6a408b7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:27.821435 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7446d58e-890d-4968-be2c-72c6d6a408b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:27.822864 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7446d58e-890d-4968-be2c-72c6d6a408b7 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:27.833491 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:27.841865 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 281/578] 10.222.0.15 () {56 vars in 1137 bytes} [Wed Aug 24 05:24:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:27.842748 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7446d58e-890d-4968-be2c-72c6d6a408b7 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:27.854816 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2462d-bb47-41c4-9616-bc3753a70d11 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:27.889200 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab9b265-eb3a-47ba-82ab-944c7c0281ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:27.890379 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab9b265-eb3a-47ba-82ab-944c7c0281ec None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:27.891128 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab9b265-eb3a-47ba-82ab-944c7c0281ec None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:27.891413 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab9b265-eb3a-47ba-82ab-944c7c0281ec None admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:27.892376 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab9b265-eb3a-47ba-82ab-944c7c0281ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:27.892376 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab9b265-eb3a-47ba-82ab-944c7c0281ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:27.893829 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab9b265-eb3a-47ba-82ab-944c7c0281ec None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:27.904851 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 298/579] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Aug 24 05:24:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:27.905727 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab9b265-eb3a-47ba-82ab-944c7c0281ec None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:27.948596 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7446d58e-890d-4968-be2c-72c6d6a408b7 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:27.972772 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:27.984651 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82badd79-ab83-4568-b1c1-0815d622c906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:27.986173 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82badd79-ab83-4568-b1c1-0815d622c906 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:27.986173 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82badd79-ab83-4568-b1c1-0815d622c906 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:27.986173 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82badd79-ab83-4568-b1c1-0815d622c906 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:27.993033 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82badd79-ab83-4568-b1c1-0815d622c906 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:27.994479 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82badd79-ab83-4568-b1c1-0815d622c906 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:28.006770 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-82badd79-ab83-4568-b1c1-0815d622c906 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 05:24:28.006770 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:28.006770 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:28.006770 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:28.006770 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:28.006770 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:28.006770 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:28.006770 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:28.006770 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:28.006770 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:28.006770 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:28.006770 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:28.006770 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:24:28.006770 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:24:28.006770 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:24:28.006770 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:24:28.007883 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:28.007883 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:28.007883 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:24:28.007883 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:28.007883 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:24:28.007883 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:28.007883 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:28.007883 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:28.007883 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:28.007883 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:28.007883 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:28.007883 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:28.007883 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:28.007883 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:28.007883 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:28.007883 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:28.007883 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:28.007883 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:28.009371 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:28.009371 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:28.009371 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:24:28.009371 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:24:28.009371 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:24:28.009371 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:24:28.009371 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:24:28.009371 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:24:28.009371 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 05:24:28.009371 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:28.010158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab9b265-eb3a-47ba-82ab-944c7c0281ec None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:28.010246 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 282/580] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:24:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:28.010246 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82badd79-ab83-4568-b1c1-0815d622c906 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:28.044568 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1655a502-8b10-47fa-a21a-91987b840498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:28.045174 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655a502-8b10-47fa-a21a-91987b840498 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:28.045459 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655a502-8b10-47fa-a21a-91987b840498 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:28.045733 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655a502-8b10-47fa-a21a-91987b840498 None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:28.046753 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1655a502-8b10-47fa-a21a-91987b840498 None admin] RBAC: Adding query filter params (name=service) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:28.047064 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1655a502-8b10-47fa-a21a-91987b840498 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:28.048472 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1655a502-8b10-47fa-a21a-91987b840498 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:28.053572 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:28.057626 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 299/581] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Aug 24 05:24:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:28.058413 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1655a502-8b10-47fa-a21a-91987b840498 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:28.113583 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82badd79-ab83-4568-b1c1-0815d622c906 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:28.147880 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aab60b-86b6-4bf1-a3a9-325d8c352ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:28.148532 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aab60b-86b6-4bf1-a3a9-325d8c352ac2 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:28.148815 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aab60b-86b6-4bf1-a3a9-325d8c352ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:28.149087 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aab60b-86b6-4bf1-a3a9-325d8c352ac2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:28.158039 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1655a502-8b10-47fa-a21a-91987b840498 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:28.161081 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aab60b-86b6-4bf1-a3a9-325d8c352ac2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:28.162541 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aab60b-86b6-4bf1-a3a9-325d8c352ac2 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:28.176870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-f9aab60b-86b6-4bf1-a3a9-325d8c352ac2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:28.176870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:28.176870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:28.176870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:28.176870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:28.176870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:28.176870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:28.176870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:28.176870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:28.176870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:28.176870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:28.176870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:28.176870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:28.176870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:28.176870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:28.176870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:28.178017 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:28.178017 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:28.178017 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:28.178017 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:28.178017 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:28.178017 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:28.178017 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:28.178017 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:28.178017 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:28.178017 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:28.178017 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:28.178017 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:28.178017 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:28.178017 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:28.178017 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:28.178017 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:28.178017 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:28.179070 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:28.179070 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:28.179070 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:28.179969 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 283/582] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:28.180663 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aab60b-86b6-4bf1-a3a9-325d8c352ac2 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:28.196748 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb721e-aacd-4eee-9c52-1ba1448afca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:28.197387 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb721e-aacd-4eee-9c52-1ba1448afca2 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:28.197666 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb721e-aacd-4eee-9c52-1ba1448afca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:28.197941 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb721e-aacd-4eee-9c52-1ba1448afca2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:28.208884 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb721e-aacd-4eee-9c52-1ba1448afca2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:28.210411 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb721e-aacd-4eee-9c52-1ba1448afca2 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:28.215287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:28.225435 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-71eb721e-aacd-4eee-9c52-1ba1448afca2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:28.225435 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:28.225435 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:28.225435 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:28.225435 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:28.225435 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:28.225435 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:28.225435 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:28.225435 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:28.225435 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:28.225435 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:28.225435 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:28.225435 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:28.225435 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:28.225435 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:28.225435 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:28.227255 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:28.227255 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:28.227255 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:28.227255 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:28.227255 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:28.227255 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:28.227255 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:28.227255 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:28.227255 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:28.227255 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:28.227255 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:28.227255 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:28.227255 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:28.227255 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:28.227255 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:28.227255 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:28.227255 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:28.228140 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:28.228140 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:28.228140 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:28.228140 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 300/583] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:28.228140 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb721e-aacd-4eee-9c52-1ba1448afca2 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:28.283116 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aab60b-86b6-4bf1-a3a9-325d8c352ac2 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:28.285427 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:28.317492 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b256c060-007d-49b7-83f1-2f2908753df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:28.318183 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256c060-007d-49b7-83f1-2f2908753df1 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:28.318540 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256c060-007d-49b7-83f1-2f2908753df1 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:28.318916 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256c060-007d-49b7-83f1-2f2908753df1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:28.319784 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b256c060-007d-49b7-83f1-2f2908753df1 None admin] RBAC: Adding query filter params (role.id=cb40cd8499f7418ca0da5ea0f1b87ff9, scope.project.id=59592706ae9442e08353158c625e9581, user.id=bb2b6cae4a004539a95fc7dd558db255) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:28.320206 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b256c060-007d-49b7-83f1-2f2908753df1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:28.321741 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b256c060-007d-49b7-83f1-2f2908753df1 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:28.327288 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 284/584] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:24:28 2022] GET /identity/v3/role_assignments?user.id=bb2b6cae4a004539a95fc7dd558db255&scope.project.id=59592706ae9442e08353158c625e9581&role.id=cb40cd8499f7418ca0da5ea0f1b87ff9 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:28.328173 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b256c060-007d-49b7-83f1-2f2908753df1 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:28.333265 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb721e-aacd-4eee-9c52-1ba1448afca2 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:28.371391 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc5e3d-1740-4b71-bc55-5be0db8a1aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:28.372638 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc5e3d-1740-4b71-bc55-5be0db8a1aa4 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:28.372997 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc5e3d-1740-4b71-bc55-5be0db8a1aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:28.373337 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc5e3d-1740-4b71-bc55-5be0db8a1aa4 None admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:28.375184 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cc5e3d-1740-4b71-bc55-5be0db8a1aa4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:28.375780 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cc5e3d-1740-4b71-bc55-5be0db8a1aa4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:28.377754 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cc5e3d-1740-4b71-bc55-5be0db8a1aa4 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:28.403303 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 301/585] 10.222.0.15 () {56 vars in 1137 bytes} [Wed Aug 24 05:24:28 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:28.403303 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc5e3d-1740-4b71-bc55-5be0db8a1aa4 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:28.439079 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:28.444156 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b256c060-007d-49b7-83f1-2f2908753df1 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:28.484240 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b539e89b-0cbd-4da6-b7cd-55783c2db2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:28.485418 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b539e89b-0cbd-4da6-b7cd-55783c2db2c1 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:28.485839 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b539e89b-0cbd-4da6-b7cd-55783c2db2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:28.486157 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b539e89b-0cbd-4da6-b7cd-55783c2db2c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:28.496946 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b539e89b-0cbd-4da6-b7cd-55783c2db2c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:28.498473 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b539e89b-0cbd-4da6-b7cd-55783c2db2c1 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:28.517611 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-b539e89b-0cbd-4da6-b7cd-55783c2db2c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:28.517611 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:28.517611 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:28.517611 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:28.517611 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:28.517611 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:28.517611 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:28.517611 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:28.517611 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:28.517611 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:28.517611 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:28.517611 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:28.517611 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:28.517611 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:28.517611 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:28.517611 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:28.518828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:28.518828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:28.518828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:28.518828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:28.518828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:28.518828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:28.518828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:28.518828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:28.518828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:28.518828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:28.518828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:28.518828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:28.518828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:28.518828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:28.518828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:28.518828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:28.518828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:28.520084 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:28.520084 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:28.520084 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:28.523534 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 285/586] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:28.524379 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b539e89b-0cbd-4da6-b7cd-55783c2db2c1 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:28.524638 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc5e3d-1740-4b71-bc55-5be0db8a1aa4 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:28.537779 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:28.564859 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891c9f2c-181f-4105-8265-f42b90fb3c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:28.565504 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c9f2c-181f-4105-8265-f42b90fb3c19 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:28.565796 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c9f2c-181f-4105-8265-f42b90fb3c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:28.566102 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c9f2c-181f-4105-8265-f42b90fb3c19 None admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:28.566758 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891c9f2c-181f-4105-8265-f42b90fb3c19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:28.567069 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891c9f2c-181f-4105-8265-f42b90fb3c19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:28.568484 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891c9f2c-181f-4105-8265-f42b90fb3c19 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:28.582844 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 302/587] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:28.584286 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891c9f2c-181f-4105-8265-f42b90fb3c19 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:28.643353 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b539e89b-0cbd-4da6-b7cd-55783c2db2c1 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:28.679821 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c7583-3672-46a2-9ab7-6e8063edfd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:28.680452 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c7583-3672-46a2-9ab7-6e8063edfd63 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:28.680737 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c7583-3672-46a2-9ab7-6e8063edfd63 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:28.681014 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c7583-3672-46a2-9ab7-6e8063edfd63 None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:28.681649 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230c7583-3672-46a2-9ab7-6e8063edfd63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:28.681955 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230c7583-3672-46a2-9ab7-6e8063edfd63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:28.683326 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230c7583-3672-46a2-9ab7-6e8063edfd63 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:28.700508 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 286/588] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:28.700508 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c7583-3672-46a2-9ab7-6e8063edfd63 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:28.712033 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:28.717129 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891c9f2c-181f-4105-8265-f42b90fb3c19 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:28.766506 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce35a1c-a8f2-47a4-8f52-868c08f547b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:28.767138 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce35a1c-a8f2-47a4-8f52-868c08f547b6 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:28.767420 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce35a1c-a8f2-47a4-8f52-868c08f547b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:28.767700 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce35a1c-a8f2-47a4-8f52-868c08f547b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:28.779599 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce35a1c-a8f2-47a4-8f52-868c08f547b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:28.781563 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce35a1c-a8f2-47a4-8f52-868c08f547b6 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:28.797819 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-8ce35a1c-a8f2-47a4-8f52-868c08f547b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:28.797819 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:28.797819 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:28.797819 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:28.797819 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:28.797819 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:28.797819 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:28.797819 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:28.797819 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:28.797819 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:28.797819 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:28.797819 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:28.797819 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:28.797819 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:28.797819 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:28.797819 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:28.798788 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:28.798788 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:28.798788 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:28.798788 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:28.798788 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:28.798788 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:28.798788 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:28.798788 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:28.798788 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:28.798788 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:28.798788 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:28.798788 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:28.798788 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:28.798788 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:28.798788 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:28.798788 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:28.798788 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:28.799668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:28.799668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:28.799668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:28.799668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 303/589] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:28.799981 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce35a1c-a8f2-47a4-8f52-868c08f547b6 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:28.820478 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c7583-3672-46a2-9ab7-6e8063edfd63 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:28.829152 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:28.859620 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4582b2-82c9-401b-95e2-7d5b2261c7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:28.860233 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4582b2-82c9-401b-95e2-7d5b2261c7ee None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:28.860549 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4582b2-82c9-401b-95e2-7d5b2261c7ee None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:28.860822 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4582b2-82c9-401b-95e2-7d5b2261c7ee None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:28.861461 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4582b2-82c9-401b-95e2-7d5b2261c7ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:28.862158 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4582b2-82c9-401b-95e2-7d5b2261c7ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:28.863164 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4582b2-82c9-401b-95e2-7d5b2261c7ee None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:28.879641 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 287/590] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:28.879641 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4582b2-82c9-401b-95e2-7d5b2261c7ee None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:28.917770 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce35a1c-a8f2-47a4-8f52-868c08f547b6 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:28.948289 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:28.956510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dae98b0-e789-4c6e-9d26-db1a32b44f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:28.973363 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae98b0-e789-4c6e-9d26-db1a32b44f6a None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:28.973363 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae98b0-e789-4c6e-9d26-db1a32b44f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:28.973363 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae98b0-e789-4c6e-9d26-db1a32b44f6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:28.973363 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dae98b0-e789-4c6e-9d26-db1a32b44f6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:28.973363 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dae98b0-e789-4c6e-9d26-db1a32b44f6a None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:28.977826 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-3dae98b0-e789-4c6e-9d26-db1a32b44f6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:28.977826 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:28.977826 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:28.977826 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:28.977826 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:28.977826 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:28.977826 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:28.977826 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:28.977826 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:28.977826 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:28.977826 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:28.977826 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:28.977826 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:24:28.977826 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:24:28.977826 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:24:28.978702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:24:28.978702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:28.978702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:28.978702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:28.978702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:28.978702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:28.978702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:28.978702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:28.978702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:28.978702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:28.978702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:28.978702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:28.978702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:28.978702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:28.978702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:28.978702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:28.978702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:28.979605 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:24:28.979605 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:24:28.979605 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:24:28.979605 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:24:28.979605 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:24:28.979605 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:24:28.979605 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:28.979605 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:28.979605 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 304/591] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:24:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:28.980132 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dae98b0-e789-4c6e-9d26-db1a32b44f6a None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:28.988576 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4582b2-82c9-401b-95e2-7d5b2261c7ee None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:29.032498 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc775fc2-1d13-4e44-af6f-d79fbf6350ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:29.033154 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc775fc2-1d13-4e44-af6f-d79fbf6350ea None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:29.033433 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc775fc2-1d13-4e44-af6f-d79fbf6350ea None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:29.033711 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc775fc2-1d13-4e44-af6f-d79fbf6350ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:29.040887 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc775fc2-1d13-4e44-af6f-d79fbf6350ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:29.042336 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc775fc2-1d13-4e44-af6f-d79fbf6350ea None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:29.055548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-fc775fc2-1d13-4e44-af6f-d79fbf6350ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:29.055548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:29.055548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:29.055548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:29.055548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:29.055548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:29.055548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:29.055548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:29.055548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:29.055548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:29.055548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:29.055548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:29.055548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:24:29.055548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:24:29.055548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:24:29.056557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:24:29.056557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:29.056557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:29.056557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:29.056557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:29.056557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:29.056557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:29.056557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:29.056557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:29.056557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:29.056557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:29.056557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:29.056557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:29.056557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:29.056557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:29.056557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:29.056557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:29.057450 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:24:29.057450 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:24:29.057450 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:24:29.057450 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:24:29.057450 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:24:29.057450 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:24:29.057450 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:29.057450 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:29.057450 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 288/592] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:24:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:29.057967 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc775fc2-1d13-4e44-af6f-d79fbf6350ea None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:29.083286 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:29.103200 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dae98b0-e789-4c6e-9d26-db1a32b44f6a None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:29.140821 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae30f3a-d6c5-4971-a7bc-f75bd4a07d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:29.141437 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae30f3a-d6c5-4971-a7bc-f75bd4a07d01 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:29.141721 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae30f3a-d6c5-4971-a7bc-f75bd4a07d01 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:29.142203 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae30f3a-d6c5-4971-a7bc-f75bd4a07d01 None admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:29.142637 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae30f3a-d6c5-4971-a7bc-f75bd4a07d01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:29.143082 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae30f3a-d6c5-4971-a7bc-f75bd4a07d01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:29.144374 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae30f3a-d6c5-4971-a7bc-f75bd4a07d01 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:29.157998 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 305/593] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:29.158855 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae30f3a-d6c5-4971-a7bc-f75bd4a07d01 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:29.191430 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc775fc2-1d13-4e44-af6f-d79fbf6350ea None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:29.228439 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:29.235338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41279c30-3cd7-45a2-941e-98c5a1ff6ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:29.236113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41279c30-3cd7-45a2-941e-98c5a1ff6ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:29.236425 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41279c30-3cd7-45a2-941e-98c5a1ff6ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:29.236691 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41279c30-3cd7-45a2-941e-98c5a1ff6ce3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:29.244717 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41279c30-3cd7-45a2-941e-98c5a1ff6ce3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:29.246196 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41279c30-3cd7-45a2-941e-98c5a1ff6ce3 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:29.257462 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-41279c30-3cd7-45a2-941e-98c5a1ff6ce3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 05:24:29.257462 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:29.257462 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:29.257462 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:29.257462 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:29.257462 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:29.257462 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:29.257462 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:29.257462 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:29.257462 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:29.257462 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:29.257462 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:29.257462 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:24:29.257462 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:24:29.257462 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:24:29.257462 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:24:29.258684 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:29.258684 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:29.258684 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:24:29.258684 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:29.258684 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:24:29.258684 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:29.258684 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:29.258684 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:29.258684 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:29.258684 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:29.258684 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:29.258684 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:29.258684 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:29.258684 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:29.258684 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:29.258684 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:29.258684 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:29.258684 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:29.259638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:29.259638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:29.259638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:24:29.259638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:24:29.259638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:24:29.259638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:24:29.259638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:24:29.259638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:24:29.259638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 05:24:29.259638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:29.268703 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 289/594] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Aug 24 05:24:29 2022] GET /identity/v3/users/neutron => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:29.269800 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41279c30-3cd7-45a2-941e-98c5a1ff6ce3 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:29.277804 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae30f3a-d6c5-4971-a7bc-f75bd4a07d01 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:29.316873 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0cb3b4-4558-476a-b19c-1229ae152353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:29.317489 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cb3b4-4558-476a-b19c-1229ae152353 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:29.317770 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cb3b4-4558-476a-b19c-1229ae152353 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:29.318044 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cb3b4-4558-476a-b19c-1229ae152353 None admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:29.319142 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0cb3b4-4558-476a-b19c-1229ae152353 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:29.319452 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0cb3b4-4558-476a-b19c-1229ae152353 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:29.321240 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0cb3b4-4558-476a-b19c-1229ae152353 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:29.334781 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 306/595] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Aug 24 05:24:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:29.335632 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0cb3b4-4558-476a-b19c-1229ae152353 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:29.338396 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:29.396048 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41279c30-3cd7-45a2-941e-98c5a1ff6ce3 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:29.433819 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff64a2-5000-40eb-a051-3ccc885f251a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:29.434422 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff64a2-5000-40eb-a051-3ccc885f251a None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:29.434701 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff64a2-5000-40eb-a051-3ccc885f251a None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:29.434980 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff64a2-5000-40eb-a051-3ccc885f251a None admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:29.435729 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ff64a2-5000-40eb-a051-3ccc885f251a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:29.435977 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ff64a2-5000-40eb-a051-3ccc885f251a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:29.438053 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ff64a2-5000-40eb-a051-3ccc885f251a None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:29.450184 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 290/596] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Aug 24 05:24:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:29.455573 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff64a2-5000-40eb-a051-3ccc885f251a None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:29.459563 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0cb3b4-4558-476a-b19c-1229ae152353 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:29.498718 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849a7292-cce2-4097-a7eb-dde4e23964e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:29.499996 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a7292-cce2-4097-a7eb-dde4e23964e1 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:29.500563 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a7292-cce2-4097-a7eb-dde4e23964e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:29.500563 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a7292-cce2-4097-a7eb-dde4e23964e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:29.507551 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849a7292-cce2-4097-a7eb-dde4e23964e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:29.509087 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849a7292-cce2-4097-a7eb-dde4e23964e1 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:29.511144 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:29.522202 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-849a7292-cce2-4097-a7eb-dde4e23964e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:29.522202 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:29.522202 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:29.522202 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:29.522202 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:29.522202 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:29.522202 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:29.522202 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:29.522202 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:29.522202 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:29.522202 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:29.522202 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:29.522202 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:24:29.522202 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:24:29.522202 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:24:29.526667 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:24:29.526667 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:29.526667 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:29.526667 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:29.526667 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:29.526667 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:29.526667 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:29.526667 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:29.526667 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:29.526667 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:29.526667 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:29.526667 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:29.526667 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:29.526667 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:29.526667 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:29.526667 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:29.526667 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:29.527637 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:24:29.527637 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:24:29.527637 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:24:29.527637 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:24:29.527637 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:24:29.527637 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:24:29.527637 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:29.527637 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:29.527637 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 307/597] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:24:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:29.527637 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849a7292-cce2-4097-a7eb-dde4e23964e1 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:29.573852 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff64a2-5000-40eb-a051-3ccc885f251a None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:29.599831 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:29.613983 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b4d156-7bee-4f7c-9b70-7da74760b77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:29.614656 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b4d156-7bee-4f7c-9b70-7da74760b77b None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:29.614939 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b4d156-7bee-4f7c-9b70-7da74760b77b None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:29.615259 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b4d156-7bee-4f7c-9b70-7da74760b77b None admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:29.616008 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b4d156-7bee-4f7c-9b70-7da74760b77b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:29.616326 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b4d156-7bee-4f7c-9b70-7da74760b77b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:29.617888 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b4d156-7bee-4f7c-9b70-7da74760b77b None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:29.637518 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 291/598] 10.222.0.15 () {56 vars in 1139 bytes} [Wed Aug 24 05:24:29 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:29.642810 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b4d156-7bee-4f7c-9b70-7da74760b77b None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:29.647038 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849a7292-cce2-4097-a7eb-dde4e23964e1 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:29.684932 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3faf07e-77b2-4b51-9b30-1bea4ce0bc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:29.685593 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3faf07e-77b2-4b51-9b30-1bea4ce0bc9b None admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:29.685932 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3faf07e-77b2-4b51-9b30-1bea4ce0bc9b None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:29.686232 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3faf07e-77b2-4b51-9b30-1bea4ce0bc9b None admin] PATH_INFO: `/v3/projects/59592706ae9442e08353158c625e9581/users/fbecffb800584c5786813accd061421b/roles/cb40cd8499f7418ca0da5ea0f1b87ff9` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:29.702198 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3faf07e-77b2-4b51-9b30-1bea4ce0bc9b None admin] RBAC: Authorizing `identity:create_grant(project_id=59592706ae9442e08353158c625e9581, user_id=fbecffb800584c5786813accd061421b, role_id=cb40cd8499f7418ca0da5ea0f1b87ff9)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:29.703661 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3faf07e-77b2-4b51-9b30-1bea4ce0bc9b None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:29.758008 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b4d156-7bee-4f7c-9b70-7da74760b77b None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:29.767414 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:29.774202 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 308/599] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Aug 24 05:24:29 2022] PUT /identity/v3/projects/59592706ae9442e08353158c625e9581/users/fbecffb800584c5786813accd061421b/roles/cb40cd8499f7418ca0da5ea0f1b87ff9 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:24:29.775334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3faf07e-77b2-4b51-9b30-1bea4ce0bc9b None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:29.807588 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add03cc-7e41-4e1f-b42d-b9a2abed712a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:29.808316 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add03cc-7e41-4e1f-b42d-b9a2abed712a None admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:29.808640 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add03cc-7e41-4e1f-b42d-b9a2abed712a None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:29.808974 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add03cc-7e41-4e1f-b42d-b9a2abed712a None admin] PATH_INFO: `/v3/projects/59592706ae9442e08353158c625e9581/users/9aee9c524a974a248c3bf363444f584a/roles/cb40cd8499f7418ca0da5ea0f1b87ff9` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:29.828070 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2add03cc-7e41-4e1f-b42d-b9a2abed712a None admin] RBAC: Authorizing `identity:create_grant(project_id=59592706ae9442e08353158c625e9581, user_id=9aee9c524a974a248c3bf363444f584a, role_id=cb40cd8499f7418ca0da5ea0f1b87ff9)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:29.829825 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2add03cc-7e41-4e1f-b42d-b9a2abed712a None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:29.909491 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3faf07e-77b2-4b51-9b30-1bea4ce0bc9b None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:29.925684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:29.937364 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 292/600] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Aug 24 05:24:29 2022] PUT /identity/v3/projects/59592706ae9442e08353158c625e9581/users/9aee9c524a974a248c3bf363444f584a/roles/cb40cd8499f7418ca0da5ea0f1b87ff9 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:24:29.937364 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add03cc-7e41-4e1f-b42d-b9a2abed712a None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:29.975774 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fdffa9-8fb4-47c0-9f82-6cd04c1bdbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:29.976478 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fdffa9-8fb4-47c0-9f82-6cd04c1bdbee None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:29.976767 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fdffa9-8fb4-47c0-9f82-6cd04c1bdbee None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:29.977538 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fdffa9-8fb4-47c0-9f82-6cd04c1bdbee None admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:29.978251 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fdffa9-8fb4-47c0-9f82-6cd04c1bdbee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:29.979394 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fdffa9-8fb4-47c0-9f82-6cd04c1bdbee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:29.981377 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fdffa9-8fb4-47c0-9f82-6cd04c1bdbee None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:29.998666 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 309/601] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Aug 24 05:24:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:30.007069 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fdffa9-8fb4-47c0-9f82-6cd04c1bdbee None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:30.086895 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add03cc-7e41-4e1f-b42d-b9a2abed712a None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:30.125852 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a39add-4de8-4e86-b11e-06b7298f0649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:30.126663 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a39add-4de8-4e86-b11e-06b7298f0649 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:30.126946 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a39add-4de8-4e86-b11e-06b7298f0649 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:30.127503 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a39add-4de8-4e86-b11e-06b7298f0649 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:30.135139 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fdffa9-8fb4-47c0-9f82-6cd04c1bdbee None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:30.141321 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a39add-4de8-4e86-b11e-06b7298f0649 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:30.142784 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a39add-4de8-4e86-b11e-06b7298f0649 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:30.161113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-11a39add-4de8-4e86-b11e-06b7298f0649 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:30.161113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:30.161113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:30.161113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:30.161113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:30.161113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:30.161113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:30.161113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:30.161113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:30.161113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:30.161113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:30.161113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:30.161113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:30.161113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:30.161113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:30.161113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:30.162085 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:30.162085 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:30.162085 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:30.162085 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:30.162085 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:30.162085 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:30.162085 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:30.162085 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:30.162085 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:30.162085 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:30.162085 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:30.162085 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:30.162085 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:30.162085 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:30.162085 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:30.162085 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:30.162085 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:30.162966 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:30.162966 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:30.162966 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:30.162966 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 293/602] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:30.168511 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a39add-4de8-4e86-b11e-06b7298f0649 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:30.173818 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b2cae-4cb9-41ca-9d5e-e2c25e9ed283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:30.174488 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b2cae-4cb9-41ca-9d5e-e2c25e9ed283 None admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:30.174764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b2cae-4cb9-41ca-9d5e-e2c25e9ed283 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:30.175095 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b2cae-4cb9-41ca-9d5e-e2c25e9ed283 None admin] PATH_INFO: `/v3/projects/59592706ae9442e08353158c625e9581/users/47bfd618ef39498f91fe7731cd40253b/roles/cb40cd8499f7418ca0da5ea0f1b87ff9` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:30.182143 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:30.204726 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b2cae-4cb9-41ca-9d5e-e2c25e9ed283 None admin] RBAC: Authorizing `identity:create_grant(project_id=59592706ae9442e08353158c625e9581, user_id=47bfd618ef39498f91fe7731cd40253b, role_id=cb40cd8499f7418ca0da5ea0f1b87ff9)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:30.204726 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b2cae-4cb9-41ca-9d5e-e2c25e9ed283 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:30.282137 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a39add-4de8-4e86-b11e-06b7298f0649 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:30.322065 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff88866e-4fa3-4880-a444-5bcf28282989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:30.322676 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff88866e-4fa3-4880-a444-5bcf28282989 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:30.322956 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff88866e-4fa3-4880-a444-5bcf28282989 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:30.323235 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff88866e-4fa3-4880-a444-5bcf28282989 None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:30.323883 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff88866e-4fa3-4880-a444-5bcf28282989 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:30.324181 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff88866e-4fa3-4880-a444-5bcf28282989 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:30.325645 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff88866e-4fa3-4880-a444-5bcf28282989 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:30.340899 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 294/603] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:24:30.340899 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a9bea-b6e2-4301-8af1-06e3e47fe46b None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:30.340899 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a9bea-b6e2-4301-8af1-06e3e47fe46b None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:30.340899 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a9bea-b6e2-4301-8af1-06e3e47fe46b None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:30.340899 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 295/604] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:30.345035 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06a9bea-b6e2-4301-8af1-06e3e47fe46b None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:30.360630 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 310/605] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Aug 24 05:24:30 2022] PUT /identity/v3/projects/59592706ae9442e08353158c625e9581/users/47bfd618ef39498f91fe7731cd40253b/roles/cb40cd8499f7418ca0da5ea0f1b87ff9 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:24:30.368478 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b970a-e77b-4a40-84c1-5c1cb2788469 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:30.368478 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b970a-e77b-4a40-84c1-5c1cb2788469 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:30.368478 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b970a-e77b-4a40-84c1-5c1cb2788469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:30.419197 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-566b970a-e77b-4a40-84c1-5c1cb2788469 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:30.423432 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-566b970a-e77b-4a40-84c1-5c1cb2788469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:30.451556 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:30.511407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:30.566681 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 311/606] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:24:30.572719 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06a9bea-b6e2-4301-8af1-06e3e47fe46b None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:30.621630 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c86149d-8245-4dbb-a7a3-276c221133a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:30.622628 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c86149d-8245-4dbb-a7a3-276c221133a0 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:30.623134 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c86149d-8245-4dbb-a7a3-276c221133a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:30.623663 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c86149d-8245-4dbb-a7a3-276c221133a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:30.632580 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c86149d-8245-4dbb-a7a3-276c221133a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:30.634400 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c86149d-8245-4dbb-a7a3-276c221133a0 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:30.647171 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-9c86149d-8245-4dbb-a7a3-276c221133a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:30.647171 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:30.647171 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:30.647171 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:30.647171 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:30.647171 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:30.647171 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:30.647171 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:30.647171 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:30.647171 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:30.647171 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:30.647171 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:30.647171 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:24:30.647171 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:24:30.647171 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:24:30.648080 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:24:30.648080 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:30.648080 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:30.648080 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:30.648080 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:30.648080 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:30.648080 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:30.648080 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:30.648080 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:30.648080 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:30.648080 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:30.648080 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:30.648080 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:30.648080 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:30.648080 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:30.648080 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:30.648080 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:30.649010 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:24:30.649010 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:24:30.649010 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:24:30.649010 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:24:30.649010 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:24:30.649010 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:24:30.649010 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:30.649010 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:30.656216 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 296/607] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:24:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:24:30.658004 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b970a-e77b-4a40-84c1-5c1cb2788469 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:30.782905 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b970a-e77b-4a40-84c1-5c1cb2788469 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:30.855710 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b94459-c19b-48a4-b211-71f28ff71b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:30.856741 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b94459-c19b-48a4-b211-71f28ff71b6d None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:30.857233 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b94459-c19b-48a4-b211-71f28ff71b6d None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:30.857798 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b94459-c19b-48a4-b211-71f28ff71b6d None admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:30.859345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b94459-c19b-48a4-b211-71f28ff71b6d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:30.859877 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b94459-c19b-48a4-b211-71f28ff71b6d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:30.861611 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b94459-c19b-48a4-b211-71f28ff71b6d None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:30.879808 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 312/608] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Aug 24 05:24:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:30.885703 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c86149d-8245-4dbb-a7a3-276c221133a0 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:30.915852 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b94459-c19b-48a4-b211-71f28ff71b6d None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:30.933977 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:31.072262 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b94459-c19b-48a4-b211-71f28ff71b6d None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:31.075168 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c86149d-8245-4dbb-a7a3-276c221133a0 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:31.116267 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:31.125417 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5edbb13-cae9-4a76-973a-214d83d345f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:31.129997 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5edbb13-cae9-4a76-973a-214d83d345f2 None admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:31.129997 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5edbb13-cae9-4a76-973a-214d83d345f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:31.129997 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5edbb13-cae9-4a76-973a-214d83d345f2 None admin] PATH_INFO: `/v3/projects/59592706ae9442e08353158c625e9581/users/b1ecd9b302b64ecf81d7e3d3356ddf7d/roles/cb40cd8499f7418ca0da5ea0f1b87ff9` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:31.147215 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5edbb13-cae9-4a76-973a-214d83d345f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=59592706ae9442e08353158c625e9581, user_id=b1ecd9b302b64ecf81d7e3d3356ddf7d, role_id=cb40cd8499f7418ca0da5ea0f1b87ff9)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:31.150185 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5edbb13-cae9-4a76-973a-214d83d345f2 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:31.154803 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b71650-0156-4121-8774-d187f822e80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:31.156336 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b71650-0156-4121-8774-d187f822e80f None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:31.156873 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b71650-0156-4121-8774-d187f822e80f None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:31.157355 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b71650-0156-4121-8774-d187f822e80f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:31.175662 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b71650-0156-4121-8774-d187f822e80f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:24:31.181236 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-34b71650-0156-4121-8774-d187f822e80f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:24:31.181236 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:31.181236 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:31.181236 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:31.181236 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:31.181236 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:31.181236 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:31.181236 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:31.181236 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:31.181236 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:31.181236 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:31.181236 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:31.181236 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:24:31.181236 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:24:31.181236 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:24:31.181236 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:24:31.182272 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:24:31.182272 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:24:31.182272 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:24:31.182272 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:24:31.182272 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:31.182272 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:31.182272 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:31.182272 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:31.182272 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:31.182272 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:31.182272 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:31.182272 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:31.182272 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:31.182272 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:31.182272 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:31.182272 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:31.183204 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:31.183204 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:31.183204 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:31.183204 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:31.183204 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:24:31.183204 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:24:31.183204 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:24:31.183204 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:24:31.183204 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:24:31.183204 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:24:31.183204 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:24:31.183204 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:31.186139 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 313/609] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Aug 24 05:24:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:31.193056 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b71650-0156-4121-8774-d187f822e80f None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:31.247096 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 297/610] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Aug 24 05:24:30 2022] PUT /identity/v3/projects/59592706ae9442e08353158c625e9581/users/b1ecd9b302b64ecf81d7e3d3356ddf7d/roles/cb40cd8499f7418ca0da5ea0f1b87ff9 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:24:31.290878 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:31.334793 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b71650-0156-4121-8774-d187f822e80f None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:31.374120 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cce7c6f-23ac-4b6b-8456-3765789b20c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:31.375177 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce7c6f-23ac-4b6b-8456-3765789b20c9 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:31.375489 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce7c6f-23ac-4b6b-8456-3765789b20c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:31.375774 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce7c6f-23ac-4b6b-8456-3765789b20c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:31.376602 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cce7c6f-23ac-4b6b-8456-3765789b20c9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:31.376918 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cce7c6f-23ac-4b6b-8456-3765789b20c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:31.378318 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cce7c6f-23ac-4b6b-8456-3765789b20c9 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:31.393190 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 314/611] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Aug 24 05:24:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:31.398989 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5edbb13-cae9-4a76-973a-214d83d345f2 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:31.529071 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5edbb13-cae9-4a76-973a-214d83d345f2 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:31.569095 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce9071-80b8-4bbe-ac44-5fc70d8760ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:31.569850 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce9071-80b8-4bbe-ac44-5fc70d8760ef None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:31.570188 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce9071-80b8-4bbe-ac44-5fc70d8760ef None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:31.570471 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce9071-80b8-4bbe-ac44-5fc70d8760ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:31.582211 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ce9071-80b8-4bbe-ac44-5fc70d8760ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:31.583972 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ce9071-80b8-4bbe-ac44-5fc70d8760ef None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:31.600452 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-c3ce9071-80b8-4bbe-ac44-5fc70d8760ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:31.600452 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:31.600452 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:31.600452 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:31.600452 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:31.600452 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:31.600452 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:31.600452 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:31.600452 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:31.600452 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:31.600452 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:31.600452 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:31.600452 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:31.600452 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:31.600452 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:31.600452 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:31.601626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:31.601626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:31.601626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:31.601626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:31.601626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:31.601626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:31.601626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:31.601626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:31.601626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:31.601626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:31.601626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:31.601626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:31.601626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:31.601626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:31.601626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:31.601626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:31.601626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:31.602709 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:31.602709 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:31.602709 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:31.604709 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 298/612] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:31.607520 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cce7c6f-23ac-4b6b-8456-3765789b20c9 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:31.729376 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cce7c6f-23ac-4b6b-8456-3765789b20c9 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:31.775338 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e342653-8608-40a9-a971-51910172a2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:31.775956 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e342653-8608-40a9-a971-51910172a2f3 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:31.776243 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e342653-8608-40a9-a971-51910172a2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:31.776597 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e342653-8608-40a9-a971-51910172a2f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:31.777692 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e342653-8608-40a9-a971-51910172a2f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:31.778008 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e342653-8608-40a9-a971-51910172a2f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:31.779383 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e342653-8608-40a9-a971-51910172a2f3 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:31.783065 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:31.797348 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 315/613] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:31.803127 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce9071-80b8-4bbe-ac44-5fc70d8760ef None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:31.856910 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:31.962001 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce9071-80b8-4bbe-ac44-5fc70d8760ef None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:32.018536 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513454a9-702c-4de8-84c4-aa3df42627ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:32.019171 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513454a9-702c-4de8-84c4-aa3df42627ba None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:32.019453 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513454a9-702c-4de8-84c4-aa3df42627ba None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:32.019725 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513454a9-702c-4de8-84c4-aa3df42627ba None admin] PATH_INFO: `/v3/users/swift` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:32.033614 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513454a9-702c-4de8-84c4-aa3df42627ba None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:32.035099 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513454a9-702c-4de8-84c4-aa3df42627ba None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:32.053904 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-513454a9-702c-4de8-84c4-aa3df42627ba None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 05:24:32.053904 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:32.053904 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:32.053904 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:32.053904 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:32.053904 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:32.053904 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:32.053904 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:32.053904 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:32.053904 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:32.053904 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:32.053904 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:32.053904 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:24:32.053904 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:24:32.053904 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:24:32.053904 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:24:32.056099 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:32.056099 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:32.056099 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:24:32.056099 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:32.056099 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:24:32.056099 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:32.056099 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:32.056099 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:32.056099 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:32.056099 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:32.056099 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:32.056099 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:32.056099 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:32.056099 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:32.056099 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:32.056099 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:32.056099 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:32.056099 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:32.058366 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:32.058366 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:32.058366 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:24:32.058366 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:24:32.058366 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:24:32.058366 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:24:32.058366 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:24:32.058366 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:24:32.058366 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 05:24:32.058366 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:32.058366 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 299/614] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Aug 24 05:24:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:32.062142 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e342653-8608-40a9-a971-51910172a2f3 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:32.272734 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e342653-8608-40a9-a971-51910172a2f3 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:32.329957 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3aa8d0-88fd-4da1-8cf9-91f09d8f6009 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:32.330304 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3aa8d0-88fd-4da1-8cf9-91f09d8f6009 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:32.330523 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3aa8d0-88fd-4da1-8cf9-91f09d8f6009 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:32.332162 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 300/615] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:32.339867 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8477b-ccfb-4e42-8e66-02122cb6b776 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:32.340156 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8477b-ccfb-4e42-8e66-02122cb6b776 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:32.340495 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8477b-ccfb-4e42-8e66-02122cb6b776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:32.341590 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42134bd-de4c-4872-9a30-cf5cc11fea29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:32.342199 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42134bd-de4c-4872-9a30-cf5cc11fea29 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:32.342480 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42134bd-de4c-4872-9a30-cf5cc11fea29 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:32.342903 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42134bd-de4c-4872-9a30-cf5cc11fea29 None admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:32.343644 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42134bd-de4c-4872-9a30-cf5cc11fea29 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:32.343952 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42134bd-de4c-4872-9a30-cf5cc11fea29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:32.345389 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42134bd-de4c-4872-9a30-cf5cc11fea29 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:32.365049 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 316/616] 10.222.0.15 () {56 vars in 1135 bytes} [Wed Aug 24 05:24:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:32.373445 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42134bd-de4c-4872-9a30-cf5cc11fea29 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:32.385437 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-1fc8477b-ccfb-4e42-8e66-02122cb6b776 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:32.386663 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-1fc8477b-ccfb-4e42-8e66-02122cb6b776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:32.459784 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:32.494944 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 301/617] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:24:32.494944 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d8539-a0b3-40a5-b81a-4873d9f368f6 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:32.494944 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d8539-a0b3-40a5-b81a-4873d9f368f6 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:32.494944 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d8539-a0b3-40a5-b81a-4873d9f368f6 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:32.500662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 302/618] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:32.503032 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df16ea-c770-446b-87c8-a0f1a055a858 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:32.503337 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df16ea-c770-446b-87c8-a0f1a055a858 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:32.503700 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df16ea-c770-446b-87c8-a0f1a055a858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:32.531750 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-67df16ea-c770-446b-87c8-a0f1a055a858 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:32.533069 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-67df16ea-c770-446b-87c8-a0f1a055a858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:32.542281 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42134bd-de4c-4872-9a30-cf5cc11fea29 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:32.572669 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:32.596915 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb8f53f-84e4-4311-8de1-6b20ee0b71cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:32.597542 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb8f53f-84e4-4311-8de1-6b20ee0b71cc None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:32.597821 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb8f53f-84e4-4311-8de1-6b20ee0b71cc None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:32.598148 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb8f53f-84e4-4311-8de1-6b20ee0b71cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:32.610950 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb8f53f-84e4-4311-8de1-6b20ee0b71cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:32.612641 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb8f53f-84e4-4311-8de1-6b20ee0b71cc None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:32.628111 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-ddb8f53f-84e4-4311-8de1-6b20ee0b71cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:32.628111 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:32.628111 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:32.628111 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:32.628111 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:32.628111 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:32.628111 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:32.628111 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:32.628111 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:32.628111 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:32.628111 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:32.628111 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:32.628111 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:32.628111 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:32.628111 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:32.628111 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:32.629313 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:32.629313 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:32.629313 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:32.629313 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:32.629313 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:32.629313 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:32.629313 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:32.629313 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:32.629313 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:32.629313 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:32.629313 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:32.629313 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:32.629313 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:32.629313 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:32.629313 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:32.629313 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:32.629313 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:32.630207 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:32.630207 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:32.630207 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:32.630207 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 317/619] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:32.630965 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878a8a7-2ebe-41c7-b639-5cf4a0ac4646 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:32.630965 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878a8a7-2ebe-41c7-b639-5cf4a0ac4646 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:32.631132 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878a8a7-2ebe-41c7-b639-5cf4a0ac4646 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:32.632585 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 318/620] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:32.636020 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878a8a7-2ebe-41c7-b639-5cf4a0ac4646 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:32.644130 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 303/621] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:24:32.646908 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053037cb-ac0b-42f2-abb1-2ddf1a4cd3cc None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:32.647207 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053037cb-ac0b-42f2-abb1-2ddf1a4cd3cc None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:32.647480 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053037cb-ac0b-42f2-abb1-2ddf1a4cd3cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:32.685553 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-053037cb-ac0b-42f2-abb1-2ddf1a4cd3cc None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:32.687358 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-053037cb-ac0b-42f2-abb1-2ddf1a4cd3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:32.779715 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878a8a7-2ebe-41c7-b639-5cf4a0ac4646 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:32.789990 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:32.857788 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16aa16c-f12e-4b3f-b73c-1c9b96697e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:32.859470 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16aa16c-f12e-4b3f-b73c-1c9b96697e39 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:32.859827 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16aa16c-f12e-4b3f-b73c-1c9b96697e39 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:32.860111 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16aa16c-f12e-4b3f-b73c-1c9b96697e39 None admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:32.860811 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16aa16c-f12e-4b3f-b73c-1c9b96697e39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:32.861225 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16aa16c-f12e-4b3f-b73c-1c9b96697e39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:32.862809 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16aa16c-f12e-4b3f-b73c-1c9b96697e39 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:32.863557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 304/622] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:24:32.864659 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053037cb-ac0b-42f2-abb1-2ddf1a4cd3cc None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:32.877259 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 319/623] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:32.882577 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16aa16c-f12e-4b3f-b73c-1c9b96697e39 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:32.977797 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:32.999624 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053037cb-ac0b-42f2-abb1-2ddf1a4cd3cc None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:33.026227 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16aa16c-f12e-4b3f-b73c-1c9b96697e39 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:33.040723 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5659f79-16df-4849-91a8-3f5607fdfaae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:33.041262 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5659f79-16df-4849-91a8-3f5607fdfaae None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:33.041595 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5659f79-16df-4849-91a8-3f5607fdfaae None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:33.041940 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5659f79-16df-4849-91a8-3f5607fdfaae None admin] PATH_INFO: `/v3/roles/service` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:33.054378 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5659f79-16df-4849-91a8-3f5607fdfaae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:24:33.059638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-a5659f79-16df-4849-91a8-3f5607fdfaae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:24:33.059638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:33.059638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:33.059638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:33.059638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:33.059638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:33.059638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:33.059638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:33.059638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:33.059638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:33.059638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:33.059638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:33.059638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:24:33.059638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:24:33.059638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:24:33.059638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:24:33.060628 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:24:33.060628 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:24:33.060628 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:24:33.060628 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:24:33.060628 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:33.060628 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:33.060628 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:33.060628 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:33.060628 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:33.060628 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:33.060628 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:33.060628 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:33.060628 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:33.060628 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:33.060628 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:33.060628 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:33.062108 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:33.062108 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:33.062108 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:33.062108 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:33.062108 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:24:33.062108 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:24:33.062108 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:24:33.062108 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:24:33.062108 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:24:33.062108 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:24:33.062108 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:24:33.062108 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:33.062108 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 305/624] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Aug 24 05:24:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:33.062806 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5659f79-16df-4849-91a8-3f5607fdfaae None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:33.066159 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6790c-0ed6-4815-8946-a1d9902c6ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:33.067068 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6790c-0ed6-4815-8946-a1d9902c6ae5 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:33.067543 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6790c-0ed6-4815-8946-a1d9902c6ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:33.068009 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6790c-0ed6-4815-8946-a1d9902c6ae5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:33.075485 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe6790c-0ed6-4815-8946-a1d9902c6ae5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:33.078364 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe6790c-0ed6-4815-8946-a1d9902c6ae5 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:33.091147 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-ebe6790c-0ed6-4815-8946-a1d9902c6ae5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:33.091147 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:33.091147 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:33.091147 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:33.091147 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:33.091147 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:33.091147 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:33.091147 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:33.091147 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:33.091147 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:33.091147 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:33.091147 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:33.091147 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:24:33.091147 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:24:33.091147 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:24:33.092062 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:24:33.092062 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:33.092062 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:33.092062 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:33.092062 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:33.092062 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:33.092062 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:33.092062 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:33.092062 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:33.092062 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:33.092062 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:33.092062 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:33.092062 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:33.092062 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:33.092062 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:33.092062 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:33.092062 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:33.093087 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:24:33.093087 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:24:33.093087 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:24:33.093087 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:24:33.093087 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:24:33.093087 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:24:33.093087 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:33.093087 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:33.099849 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 320/625] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:24:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:24:33.101014 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6790c-0ed6-4815-8946-a1d9902c6ae5 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:33.127495 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:33.181782 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5659f79-16df-4849-91a8-3f5607fdfaae None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:33.207036 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6790c-0ed6-4815-8946-a1d9902c6ae5 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:33.221024 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8077d7-5bd0-4e29-8586-2b39b711e683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:33.221664 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8077d7-5bd0-4e29-8586-2b39b711e683 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:33.221948 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8077d7-5bd0-4e29-8586-2b39b711e683 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:33.222224 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8077d7-5bd0-4e29-8586-2b39b711e683 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:33.232831 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8077d7-5bd0-4e29-8586-2b39b711e683 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:24:33.238662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-4f8077d7-5bd0-4e29-8586-2b39b711e683 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:24:33.238662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:33.238662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:33.238662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:33.238662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:33.238662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:33.238662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:33.238662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:33.238662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:33.238662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:33.238662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:33.238662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:33.238662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:24:33.238662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:24:33.238662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:24:33.238662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:24:33.239601 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:24:33.239601 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:24:33.239601 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:24:33.239601 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:24:33.239601 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:33.239601 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:33.239601 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:33.239601 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:33.239601 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:33.239601 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:33.239601 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:33.239601 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:33.239601 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:33.239601 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:33.239601 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:33.239601 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:33.240508 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:33.240508 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:33.240508 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:33.240508 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:33.240508 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:24:33.240508 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:24:33.240508 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:24:33.240508 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:24:33.240508 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:24:33.240508 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:24:33.240508 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:24:33.240508 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:33.240508 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 306/626] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Aug 24 05:24:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:33.241196 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8077d7-5bd0-4e29-8586-2b39b711e683 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:33.245427 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:33.250986 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b91028b-1698-4b51-9932-9cbd1e7a03a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:33.251611 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b91028b-1698-4b51-9932-9cbd1e7a03a8 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:33.251958 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b91028b-1698-4b51-9932-9cbd1e7a03a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:33.252234 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b91028b-1698-4b51-9932-9cbd1e7a03a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:33.252860 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b91028b-1698-4b51-9932-9cbd1e7a03a8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:33.253161 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b91028b-1698-4b51-9932-9cbd1e7a03a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:33.257607 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b91028b-1698-4b51-9932-9cbd1e7a03a8 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:33.263123 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 321/627] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Aug 24 05:24:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:33.267747 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b91028b-1698-4b51-9932-9cbd1e7a03a8 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:33.348555 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8077d7-5bd0-4e29-8586-2b39b711e683 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:33.360437 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:33.376288 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b91028b-1698-4b51-9932-9cbd1e7a03a8 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:33.389268 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be5bd49-0ea1-44ad-80c3-918c9f0cdd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:33.390475 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be5bd49-0ea1-44ad-80c3-918c9f0cdd68 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:33.391370 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be5bd49-0ea1-44ad-80c3-918c9f0cdd68 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:33.392096 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be5bd49-0ea1-44ad-80c3-918c9f0cdd68 None admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:33.393019 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be5bd49-0ea1-44ad-80c3-918c9f0cdd68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:33.393640 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be5bd49-0ea1-44ad-80c3-918c9f0cdd68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:33.395623 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be5bd49-0ea1-44ad-80c3-918c9f0cdd68 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:33.409816 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 307/628] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Aug 24 05:24:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:33.411000 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be5bd49-0ea1-44ad-80c3-918c9f0cdd68 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:33.415383 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4e3853-3802-4c0c-9235-f0a62ceb009f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:33.416024 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e3853-3802-4c0c-9235-f0a62ceb009f None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:33.416307 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e3853-3802-4c0c-9235-f0a62ceb009f None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:33.416607 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e3853-3802-4c0c-9235-f0a62ceb009f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:33.429808 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4e3853-3802-4c0c-9235-f0a62ceb009f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:24:33.434486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-fc4e3853-3802-4c0c-9235-f0a62ceb009f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:24:33.434486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:33.434486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:33.434486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:33.434486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:33.434486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:33.434486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:33.434486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:33.434486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:33.434486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:33.434486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:33.434486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:33.434486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:24:33.434486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:24:33.434486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:24:33.434486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:24:33.435445 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:24:33.435445 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:24:33.435445 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:24:33.435445 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:24:33.435445 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:33.435445 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:33.435445 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:33.435445 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:33.435445 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:33.435445 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:33.435445 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:33.435445 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:33.435445 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:33.435445 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:33.435445 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:33.435445 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:33.436327 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:33.436327 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:33.436327 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:33.436327 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:33.436327 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:24:33.436327 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:24:33.436327 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:24:33.436327 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:24:33.436327 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:24:33.436327 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:24:33.436327 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:24:33.436327 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:33.436327 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 322/629] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Aug 24 05:24:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:33.437273 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4e3853-3802-4c0c-9235-f0a62ceb009f None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:33.506966 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:33.551292 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4e3853-3802-4c0c-9235-f0a62ceb009f None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:33.551818 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be5bd49-0ea1-44ad-80c3-918c9f0cdd68 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:33.587827 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc488e20-3b22-41cc-80dc-77cd07afa0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:33.588468 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc488e20-3b22-41cc-80dc-77cd07afa0c2 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:33.588778 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc488e20-3b22-41cc-80dc-77cd07afa0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:33.589031 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc488e20-3b22-41cc-80dc-77cd07afa0c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:33.589727 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc488e20-3b22-41cc-80dc-77cd07afa0c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:33.589916 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44630564-df3a-45ff-824b-e028571c0d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:33.590048 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc488e20-3b22-41cc-80dc-77cd07afa0c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:33.590553 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44630564-df3a-45ff-824b-e028571c0d74 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:33.590837 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44630564-df3a-45ff-824b-e028571c0d74 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:33.591107 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44630564-df3a-45ff-824b-e028571c0d74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:33.591414 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc488e20-3b22-41cc-80dc-77cd07afa0c2 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:33.604547 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 308/630] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Aug 24 05:24:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:33.604547 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc488e20-3b22-41cc-80dc-77cd07afa0c2 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:33.607449 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44630564-df3a-45ff-824b-e028571c0d74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:33.609979 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44630564-df3a-45ff-824b-e028571c0d74 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:33.618926 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:33.627850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-44630564-df3a-45ff-824b-e028571c0d74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:33.627850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:33.627850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:33.627850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:33.627850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:33.627850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:33.627850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:33.627850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:33.627850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:33.627850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:33.627850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:33.627850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:33.627850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:33.627850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:33.627850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:33.627850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:33.631763 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:33.631763 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:33.631763 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:33.631763 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:33.631763 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:33.631763 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:33.631763 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:33.631763 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:33.631763 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:33.631763 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:33.631763 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:33.631763 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:33.631763 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:33.631763 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:33.631763 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:33.631763 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:33.631763 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:33.632685 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:33.632685 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:33.632685 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:33.632685 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 323/631] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:33.632685 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44630564-df3a-45ff-824b-e028571c0d74 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:33.717432 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc488e20-3b22-41cc-80dc-77cd07afa0c2 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:33.741148 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:33.745145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44630564-df3a-45ff-824b-e028571c0d74 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:33.752061 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5f4d24-14f4-4af3-ae9c-235bd4d92c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:33.752759 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f4d24-14f4-4af3-ae9c-235bd4d92c82 None admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:33.753028 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f4d24-14f4-4af3-ae9c-235bd4d92c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:33.753301 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f4d24-14f4-4af3-ae9c-235bd4d92c82 None admin] PATH_INFO: `/v3/projects/59592706ae9442e08353158c625e9581/users/bb2b6cae4a004539a95fc7dd558db255/roles/cb40cd8499f7418ca0da5ea0f1b87ff9` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:33.768417 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5f4d24-14f4-4af3-ae9c-235bd4d92c82 None admin] RBAC: Authorizing `identity:create_grant(project_id=59592706ae9442e08353158c625e9581, user_id=bb2b6cae4a004539a95fc7dd558db255, role_id=cb40cd8499f7418ca0da5ea0f1b87ff9)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:33.769915 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5f4d24-14f4-4af3-ae9c-235bd4d92c82 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:33.779149 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0681a69d-39ba-4666-9de1-a130b73e9c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:33.779677 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0681a69d-39ba-4666-9de1-a130b73e9c1b None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:33.779955 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0681a69d-39ba-4666-9de1-a130b73e9c1b None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:33.781067 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0681a69d-39ba-4666-9de1-a130b73e9c1b None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:33.781739 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0681a69d-39ba-4666-9de1-a130b73e9c1b None admin] RBAC: Adding query filter params (name=service) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:33.782041 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0681a69d-39ba-4666-9de1-a130b73e9c1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:33.783398 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0681a69d-39ba-4666-9de1-a130b73e9c1b None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:33.791759 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 324/632] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Aug 24 05:24:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:33.792583 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0681a69d-39ba-4666-9de1-a130b73e9c1b None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:33.848620 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 309/633] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Aug 24 05:24:33 2022] PUT /identity/v3/projects/59592706ae9442e08353158c625e9581/users/bb2b6cae4a004539a95fc7dd558db255/roles/cb40cd8499f7418ca0da5ea0f1b87ff9 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:24:33.849473 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5f4d24-14f4-4af3-ae9c-235bd4d92c82 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:33.864809 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:33.915771 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0681a69d-39ba-4666-9de1-a130b73e9c1b None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:33.954819 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be1b6d9-74c9-406b-9e2e-22d795def476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:33.955535 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be1b6d9-74c9-406b-9e2e-22d795def476 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:33.955821 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be1b6d9-74c9-406b-9e2e-22d795def476 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:33.956098 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be1b6d9-74c9-406b-9e2e-22d795def476 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:33.968771 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be1b6d9-74c9-406b-9e2e-22d795def476 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:33.970267 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be1b6d9-74c9-406b-9e2e-22d795def476 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:33.986905 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-2be1b6d9-74c9-406b-9e2e-22d795def476 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:33.986905 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:33.986905 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:33.986905 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:33.986905 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:33.986905 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:33.986905 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:33.986905 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:33.986905 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:33.986905 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:33.986905 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:33.986905 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:33.986905 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:33.986905 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:33.986905 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:33.986905 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:33.995055 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:33.995055 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:33.995055 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:33.995055 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:33.995055 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:33.995055 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:33.995055 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:33.995055 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:33.995055 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:33.995055 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:33.995055 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:33.995055 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:33.995055 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:33.995055 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:33.995055 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:33.995055 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:33.995055 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:33.995953 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:33.995953 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:33.995953 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:33.996134 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5f4d24-14f4-4af3-ae9c-235bd4d92c82 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:33.996219 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 325/634] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:33.996219 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2167d311-2188-4188-bc8f-1e63784346ca None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:33.996219 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2167d311-2188-4188-bc8f-1e63784346ca None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:33.996219 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2167d311-2188-4188-bc8f-1e63784346ca None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:33.996219 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 326/635] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:33.996219 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2167d311-2188-4188-bc8f-1e63784346ca None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:34.037775 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e491bf6-3014-4be1-a54a-ad2d0fe51faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:34.038390 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e491bf6-3014-4be1-a54a-ad2d0fe51faf None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:34.038670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e491bf6-3014-4be1-a54a-ad2d0fe51faf None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:34.039106 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e491bf6-3014-4be1-a54a-ad2d0fe51faf None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:34.039584 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e491bf6-3014-4be1-a54a-ad2d0fe51faf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:34.039947 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e491bf6-3014-4be1-a54a-ad2d0fe51faf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:34.041348 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e491bf6-3014-4be1-a54a-ad2d0fe51faf None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:34.054423 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 310/636] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:34.055055 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e491bf6-3014-4be1-a54a-ad2d0fe51faf None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:34.089866 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:34.124945 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2167d311-2188-4188-bc8f-1e63784346ca None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:34.139981 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:34.161408 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e491bf6-3014-4be1-a54a-ad2d0fe51faf None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:34.165915 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b69427e-bff6-474a-bf5d-65a3ec8cf3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:34.166571 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b69427e-bff6-474a-bf5d-65a3ec8cf3e4 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:34.166856 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b69427e-bff6-474a-bf5d-65a3ec8cf3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:34.167135 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b69427e-bff6-474a-bf5d-65a3ec8cf3e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:34.181990 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b69427e-bff6-474a-bf5d-65a3ec8cf3e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:34.184385 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b69427e-bff6-474a-bf5d-65a3ec8cf3e4 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:34.211349 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-7b69427e-bff6-474a-bf5d-65a3ec8cf3e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:34.211349 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:34.211349 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:34.211349 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:34.211349 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:34.211349 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:34.211349 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:34.211349 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:34.211349 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:34.211349 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:34.211349 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:34.211349 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:34.211349 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:34.211349 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:34.211349 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:34.211349 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:34.212651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:34.212651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:34.212651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:34.212651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:34.212651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:34.212651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:34.212651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:34.212651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:34.212651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:34.212651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:34.212651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:34.212651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:34.212651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:34.212651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:34.212651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:34.212651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:34.212651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:34.213755 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:34.213755 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:34.213755 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:34.215979 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 327/637] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:34.219845 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246b964-6d69-4f58-a76a-cbe7c4ede257 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:34.220327 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246b964-6d69-4f58-a76a-cbe7c4ede257 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:34.220774 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246b964-6d69-4f58-a76a-cbe7c4ede257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:34.227065 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f4b9ed-f091-4474-809c-2933178ee1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:34.227887 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4b9ed-f091-4474-809c-2933178ee1f4 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:34.228170 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4b9ed-f091-4474-809c-2933178ee1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:34.228894 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4b9ed-f091-4474-809c-2933178ee1f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:34.229622 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f4b9ed-f091-4474-809c-2933178ee1f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:34.229622 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f4b9ed-f091-4474-809c-2933178ee1f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:34.230983 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f4b9ed-f091-4474-809c-2933178ee1f4 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:34.244958 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 311/638] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:34.246092 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-3246b964-6d69-4f58-a76a-cbe7c4ede257 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:34.247447 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-3246b964-6d69-4f58-a76a-cbe7c4ede257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:34.252258 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f4b9ed-f091-4474-809c-2933178ee1f4 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:34.333033 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 328/639] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:24:34.334488 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3246b964-6d69-4f58-a76a-cbe7c4ede257 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:34.368829 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:34.383701 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f4b9ed-f091-4474-809c-2933178ee1f4 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:34.445102 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:34.454671 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3246b964-6d69-4f58-a76a-cbe7c4ede257 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:34.470423 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a444720-cc80-4a31-9c12-c9df1e018429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:34.471078 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a444720-cc80-4a31-9c12-c9df1e018429 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:34.471406 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a444720-cc80-4a31-9c12-c9df1e018429 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:34.471682 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a444720-cc80-4a31-9c12-c9df1e018429 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:34.491245 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a444720-cc80-4a31-9c12-c9df1e018429 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:34.492760 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a444720-cc80-4a31-9c12-c9df1e018429 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:34.496924 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e592f6fd-eb98-4d9d-b756-377d1b61d007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:34.497657 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592f6fd-eb98-4d9d-b756-377d1b61d007 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:34.497882 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592f6fd-eb98-4d9d-b756-377d1b61d007 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:34.498159 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592f6fd-eb98-4d9d-b756-377d1b61d007 None admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:34.499578 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e592f6fd-eb98-4d9d-b756-377d1b61d007 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:34.499789 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e592f6fd-eb98-4d9d-b756-377d1b61d007 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:34.501348 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e592f6fd-eb98-4d9d-b756-377d1b61d007 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:34.506555 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-8a444720-cc80-4a31-9c12-c9df1e018429 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:24:34.506555 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:34.506555 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:34.506555 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:34.506555 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:34.506555 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:34.506555 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:34.506555 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:34.506555 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:34.506555 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:34.506555 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:34.506555 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:34.506555 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:24:34.506555 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:24:34.506555 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:24:34.506555 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:24:34.507937 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:34.507937 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:34.507937 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:24:34.507937 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:34.507937 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:24:34.507937 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:34.507937 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:34.507937 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:34.507937 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:34.507937 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:34.507937 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:34.507937 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:34.507937 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:34.507937 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:34.507937 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:34.507937 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:34.507937 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:34.507937 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:34.508976 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:34.508976 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:34.508976 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:24:34.508976 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:24:34.508976 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:24:34.508976 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:24:34.508976 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:24:34.508976 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:24:34.508976 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:24:34.508976 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:34.508976 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 312/640] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Aug 24 05:24:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:34.515783 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a444720-cc80-4a31-9c12-c9df1e018429 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:34.518165 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 329/641] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:34.519085 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e592f6fd-eb98-4d9d-b756-377d1b61d007 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:34.675435 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e592f6fd-eb98-4d9d-b756-377d1b61d007 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:34.692200 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a444720-cc80-4a31-9c12-c9df1e018429 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:34.714946 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d0a39a-664c-4174-9059-072f12ec07f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:34.715546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d0a39a-664c-4174-9059-072f12ec07f0 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:34.715836 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d0a39a-664c-4174-9059-072f12ec07f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:34.716113 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d0a39a-664c-4174-9059-072f12ec07f0 None admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:34.716986 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d0a39a-664c-4174-9059-072f12ec07f0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:34.717295 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d0a39a-664c-4174-9059-072f12ec07f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:34.719435 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d0a39a-664c-4174-9059-072f12ec07f0 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:34.728366 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b3270-c7d6-4982-b6fd-f7379ea59fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:34.729141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b3270-c7d6-4982-b6fd-f7379ea59fae None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:34.729510 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b3270-c7d6-4982-b6fd-f7379ea59fae None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:34.729878 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b3270-c7d6-4982-b6fd-f7379ea59fae None admin] PATH_INFO: `/v3/users/glance` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:34.737733 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 330/642] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Aug 24 05:24:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:34.738633 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d0a39a-664c-4174-9059-072f12ec07f0 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:34.742409 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b3270-c7d6-4982-b6fd-f7379ea59fae None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:34.744091 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b3270-c7d6-4982-b6fd-f7379ea59fae None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:34.754600 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:34.757425 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-bc1b3270-c7d6-4982-b6fd-f7379ea59fae None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 05:24:34.757425 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:34.757425 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:34.757425 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:34.757425 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:34.757425 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:34.757425 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:34.757425 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:34.757425 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:34.757425 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:34.757425 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:34.757425 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:34.757425 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:24:34.757425 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:24:34.757425 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:24:34.757425 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:24:34.758597 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:34.758597 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:34.758597 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:24:34.758597 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:34.758597 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:24:34.758597 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:34.758597 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:34.758597 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:34.758597 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:34.758597 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:34.758597 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:34.758597 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:34.758597 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:34.758597 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:34.758597 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:34.758597 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:34.758597 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:34.758597 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:34.759600 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:34.759600 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:34.759600 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:24:34.759600 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:24:34.759600 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:24:34.759600 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:24:34.759600 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:24:34.759600 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:24:34.759600 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 05:24:34.759600 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:34.760151 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 313/643] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:24:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:34.761118 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b3270-c7d6-4982-b6fd-f7379ea59fae None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:34.778078 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:34.864437 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d0a39a-664c-4174-9059-072f12ec07f0 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:34.877380 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b3270-c7d6-4982-b6fd-f7379ea59fae None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:34.908010 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda0bc4-e6e2-430e-b87a-93559ae00379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:34.909469 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda0bc4-e6e2-430e-b87a-93559ae00379 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:34.910209 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda0bc4-e6e2-430e-b87a-93559ae00379 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:34.910670 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda0bc4-e6e2-430e-b87a-93559ae00379 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:34.914138 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3298d5-a2ac-4bf8-8f1a-6e771ecb24cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:34.914764 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3298d5-a2ac-4bf8-8f1a-6e771ecb24cc None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:34.915043 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3298d5-a2ac-4bf8-8f1a-6e771ecb24cc None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:34.915314 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3298d5-a2ac-4bf8-8f1a-6e771ecb24cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:34.919737 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dda0bc4-e6e2-430e-b87a-93559ae00379 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:34.922396 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dda0bc4-e6e2-430e-b87a-93559ae00379 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:34.926472 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3298d5-a2ac-4bf8-8f1a-6e771ecb24cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:24:34.930983 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-cc3298d5-a2ac-4bf8-8f1a-6e771ecb24cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:24:34.930983 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:34.930983 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:34.930983 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:34.930983 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:34.930983 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:34.930983 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:34.930983 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:34.930983 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:34.930983 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:34.930983 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:34.930983 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:34.930983 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:24:34.930983 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:24:34.930983 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:24:34.930983 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:24:34.931893 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:24:34.931893 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:24:34.931893 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:24:34.931893 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:24:34.931893 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:34.931893 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:34.931893 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:34.931893 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:34.931893 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:34.931893 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:34.931893 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:34.931893 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:34.931893 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:34.931893 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:34.931893 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:34.931893 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:34.932952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:34.932952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:34.932952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:34.932952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:34.932952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:24:34.932952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:24:34.932952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:24:34.932952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:24:34.932952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:24:34.932952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:24:34.932952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:24:34.932952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:34.932952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 314/644] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Aug 24 05:24:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:34.932952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3298d5-a2ac-4bf8-8f1a-6e771ecb24cc None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:34.939349 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-3dda0bc4-e6e2-430e-b87a-93559ae00379 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 05:24:34.939349 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:34.939349 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:34.939349 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:34.939349 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:34.939349 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:34.939349 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:34.939349 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:34.939349 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:34.939349 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:34.939349 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:34.939349 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:34.939349 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:24:34.939349 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:24:34.939349 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:24:34.939349 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:24:34.940279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:34.940279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:34.940279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:24:34.940279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:34.940279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:24:34.940279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:34.940279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:34.940279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:34.940279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:34.940279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:34.940279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:34.940279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:34.940279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:34.940279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:34.940279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:34.940279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:34.940279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:34.940279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:34.941729 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:34.941729 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:34.941729 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:24:34.941729 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:24:34.941729 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:24:34.941729 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:24:34.941729 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:24:34.941729 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:24:34.941729 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 05:24:34.941729 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:34.941729 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 331/645] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:24:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:34.942388 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda0bc4-e6e2-430e-b87a-93559ae00379 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:35.022120 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:35.025429 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:35.047813 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3298d5-a2ac-4bf8-8f1a-6e771ecb24cc None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:35.056815 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda0bc4-e6e2-430e-b87a-93559ae00379 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:35.083466 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaff0d4-9515-4274-8306-c1484ec2e7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:35.084069 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaff0d4-9515-4274-8306-c1484ec2e7a9 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:35.084346 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaff0d4-9515-4274-8306-c1484ec2e7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:35.084645 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaff0d4-9515-4274-8306-c1484ec2e7a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:35.093484 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c176744-7823-4a95-94ac-7964c4a6ae2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:35.094180 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c176744-7823-4a95-94ac-7964c4a6ae2e None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:35.094401 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c176744-7823-4a95-94ac-7964c4a6ae2e None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:35.094640 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c176744-7823-4a95-94ac-7964c4a6ae2e None admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:35.095369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c176744-7823-4a95-94ac-7964c4a6ae2e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:35.095789 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c176744-7823-4a95-94ac-7964c4a6ae2e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:35.097242 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c176744-7823-4a95-94ac-7964c4a6ae2e None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:35.119885 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 332/646] 10.222.0.15 () {56 vars in 1137 bytes} [Wed Aug 24 05:24:34 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:35.120603 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c176744-7823-4a95-94ac-7964c4a6ae2e None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:35.156878 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaff0d4-9515-4274-8306-c1484ec2e7a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:35.158785 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaff0d4-9515-4274-8306-c1484ec2e7a9 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:35.228334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c176744-7823-4a95-94ac-7964c4a6ae2e None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:35.257581 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:35.261959 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:35.267977 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbc912b-327a-43e2-96b7-d5dcd4edda23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:35.270492 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc912b-327a-43e2-96b7-d5dcd4edda23 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:35.270492 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc912b-327a-43e2-96b7-d5dcd4edda23 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:35.270492 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc912b-327a-43e2-96b7-d5dcd4edda23 None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:35.271236 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbc912b-327a-43e2-96b7-d5dcd4edda23 None admin] RBAC: Adding query filter params (name=service) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:35.271512 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbc912b-327a-43e2-96b7-d5dcd4edda23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:35.272962 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbc912b-327a-43e2-96b7-d5dcd4edda23 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:35.287124 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 333/647] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Aug 24 05:24:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:35.287124 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbc912b-327a-43e2-96b7-d5dcd4edda23 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:35.291884 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 315/648] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:24:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:24:35.297029 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaff0d4-9515-4274-8306-c1484ec2e7a9 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:35.406212 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaff0d4-9515-4274-8306-c1484ec2e7a9 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:35.410981 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbc912b-327a-43e2-96b7-d5dcd4edda23 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:35.444159 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd7f93-67a2-49a6-93af-791701ecbff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:35.444704 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd7f93-67a2-49a6-93af-791701ecbff4 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:35.444983 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd7f93-67a2-49a6-93af-791701ecbff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:35.445256 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd7f93-67a2-49a6-93af-791701ecbff4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:35.462831 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa81ac2-bdad-4e18-8557-9cc72dc26604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:35.463541 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa81ac2-bdad-4e18-8557-9cc72dc26604 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:35.463889 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa81ac2-bdad-4e18-8557-9cc72dc26604 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:35.464230 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa81ac2-bdad-4e18-8557-9cc72dc26604 None admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:35.465135 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa81ac2-bdad-4e18-8557-9cc72dc26604 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:35.465456 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa81ac2-bdad-4e18-8557-9cc72dc26604 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:35.466984 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa81ac2-bdad-4e18-8557-9cc72dc26604 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:35.493079 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 334/649] 10.222.0.15 () {56 vars in 1137 bytes} [Wed Aug 24 05:24:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:35.493079 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa81ac2-bdad-4e18-8557-9cc72dc26604 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:35.497666 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:35.531160 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fd7f93-67a2-49a6-93af-791701ecbff4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:35.533850 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fd7f93-67a2-49a6-93af-791701ecbff4 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:35.563592 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:35.633680 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa81ac2-bdad-4e18-8557-9cc72dc26604 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:35.659553 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 316/650] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:24:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:24:35.660456 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd7f93-67a2-49a6-93af-791701ecbff4 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:35.682406 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e8d8f-fa9e-4d5f-9c10-d67b72b626e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:35.684299 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e8d8f-fa9e-4d5f-9c10-d67b72b626e0 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:35.684817 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e8d8f-fa9e-4d5f-9c10-d67b72b626e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:35.685287 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e8d8f-fa9e-4d5f-9c10-d67b72b626e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:35.701771 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e8d8f-fa9e-4d5f-9c10-d67b72b626e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:35.703293 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e8d8f-fa9e-4d5f-9c10-d67b72b626e0 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:35.726423 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-1c5e8d8f-fa9e-4d5f-9c10-d67b72b626e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:35.726423 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:35.726423 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:35.726423 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:35.726423 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:35.726423 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:35.726423 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:35.726423 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:35.726423 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:35.726423 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:35.726423 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:35.726423 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:35.726423 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:35.726423 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:35.726423 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:35.726423 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:35.728811 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:35.728811 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:35.728811 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:35.728811 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:35.728811 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:35.728811 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:35.728811 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:35.728811 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:35.728811 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:35.728811 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:35.728811 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:35.728811 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:35.728811 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:35.728811 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:35.728811 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:35.728811 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:35.728811 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:35.729741 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:35.729741 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:35.729741 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:35.729741 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 335/651] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:35.729741 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e8d8f-fa9e-4d5f-9c10-d67b72b626e0 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:35.761490 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:35.793160 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd7f93-67a2-49a6-93af-791701ecbff4 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:35.841025 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f5c1af-34e7-46f0-aa3b-e5806c25ee5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:35.841817 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5c1af-34e7-46f0-aa3b-e5806c25ee5f None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:35.841984 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5c1af-34e7-46f0-aa3b-e5806c25ee5f None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:35.842253 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5c1af-34e7-46f0-aa3b-e5806c25ee5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:35.861231 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f5c1af-34e7-46f0-aa3b-e5806c25ee5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:35.863282 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f5c1af-34e7-46f0-aa3b-e5806c25ee5f None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:35.868323 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e8d8f-fa9e-4d5f-9c10-d67b72b626e0 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:35.871138 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:35.894495 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-f5f5c1af-34e7-46f0-aa3b-e5806c25ee5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:35.894495 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:35.894495 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:35.894495 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:35.894495 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:35.894495 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:35.894495 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:35.894495 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:35.894495 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:35.894495 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:35.894495 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:35.894495 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:35.894495 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:35.894495 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:35.894495 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:35.894495 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:35.895509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:35.895509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:35.895509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:35.895509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:35.895509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:35.895509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:35.895509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:35.895509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:35.895509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:35.895509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:35.895509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:35.895509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:35.895509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:35.895509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:35.895509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:35.895509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:35.895509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:35.896430 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:35.896430 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:35.896430 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:35.896607 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 317/652] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:35.897307 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f5c1af-34e7-46f0-aa3b-e5806c25ee5f None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:35.918928 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df8a2b7-7698-47f4-aced-7914805dedaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:35.919595 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df8a2b7-7698-47f4-aced-7914805dedaf None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:35.919871 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df8a2b7-7698-47f4-aced-7914805dedaf None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:35.920140 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df8a2b7-7698-47f4-aced-7914805dedaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:36.025805 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df8a2b7-7698-47f4-aced-7914805dedaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:36.027220 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df8a2b7-7698-47f4-aced-7914805dedaf None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:36.032192 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f5c1af-34e7-46f0-aa3b-e5806c25ee5f None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:36.063042 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:36.072772 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4999bca-6222-4c22-84da-a984614aa926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:36.073520 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4999bca-6222-4c22-84da-a984614aa926 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:36.073807 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4999bca-6222-4c22-84da-a984614aa926 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:36.088466 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4999bca-6222-4c22-84da-a984614aa926 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:36.088466 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4999bca-6222-4c22-84da-a984614aa926 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:36.088466 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4999bca-6222-4c22-84da-a984614aa926 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:36.104075 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-b4999bca-6222-4c22-84da-a984614aa926 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:36.104075 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:36.104075 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:36.104075 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:36.104075 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:36.104075 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:36.104075 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:36.104075 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:36.104075 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:36.104075 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:36.104075 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:36.104075 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:36.104075 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:36.104075 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:36.104075 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:36.104075 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:36.111314 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:36.111314 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:36.111314 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:36.111314 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:36.111314 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:36.111314 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:36.111314 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:36.111314 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:36.111314 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:36.111314 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:36.111314 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:36.111314 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:36.111314 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:36.111314 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:36.111314 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:36.111314 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:36.111314 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:36.112495 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:36.112495 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:36.112495 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:36.112495 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 318/653] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:36.112495 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4999bca-6222-4c22-84da-a984614aa926 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:36.160862 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 336/654] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:24:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:24:36.161727 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df8a2b7-7698-47f4-aced-7914805dedaf None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:36.164148 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:36.239087 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4999bca-6222-4c22-84da-a984614aa926 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:36.276295 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f66ad-7bc5-4f7a-932f-7a1d62fe474d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:36.276886 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f66ad-7bc5-4f7a-932f-7a1d62fe474d None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:36.277167 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f66ad-7bc5-4f7a-932f-7a1d62fe474d None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:36.277443 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f66ad-7bc5-4f7a-932f-7a1d62fe474d None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:36.278091 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02f66ad-7bc5-4f7a-932f-7a1d62fe474d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:36.278393 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02f66ad-7bc5-4f7a-932f-7a1d62fe474d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:36.279888 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02f66ad-7bc5-4f7a-932f-7a1d62fe474d None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:36.285027 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df8a2b7-7698-47f4-aced-7914805dedaf None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:36.296087 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 319/655] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:36.298245 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2043f-fce1-45f9-b3a7-0b7cf5fedf1c None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:36.298693 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2043f-fce1-45f9-b3a7-0b7cf5fedf1c None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:36.299151 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2043f-fce1-45f9-b3a7-0b7cf5fedf1c None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:36.301013 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 320/656] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:36.301747 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e2043f-fce1-45f9-b3a7-0b7cf5fedf1c None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:36.321934 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77dbb9e-577e-409c-bada-bc0cfce5fbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:36.322596 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77dbb9e-577e-409c-bada-bc0cfce5fbf2 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:36.322874 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77dbb9e-577e-409c-bada-bc0cfce5fbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:36.323143 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77dbb9e-577e-409c-bada-bc0cfce5fbf2 None admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:36.323780 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77dbb9e-577e-409c-bada-bc0cfce5fbf2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:36.324087 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77dbb9e-577e-409c-bada-bc0cfce5fbf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:36.325547 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77dbb9e-577e-409c-bada-bc0cfce5fbf2 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:36.338470 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 337/657] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:36.339381 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77dbb9e-577e-409c-bada-bc0cfce5fbf2 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:36.358001 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:36.410956 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:36.429730 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e2043f-fce1-45f9-b3a7-0b7cf5fedf1c None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:36.450407 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77dbb9e-577e-409c-bada-bc0cfce5fbf2 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:36.463115 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caddd393-2175-4632-80a3-108255c07d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:36.463715 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caddd393-2175-4632-80a3-108255c07d03 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:36.463993 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caddd393-2175-4632-80a3-108255c07d03 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:36.464264 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caddd393-2175-4632-80a3-108255c07d03 None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:36.464935 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caddd393-2175-4632-80a3-108255c07d03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:36.465234 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caddd393-2175-4632-80a3-108255c07d03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:36.466588 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caddd393-2175-4632-80a3-108255c07d03 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:36.478288 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 321/658] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:36.479098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caddd393-2175-4632-80a3-108255c07d03 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:36.487033 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c59e76-697f-4d5f-bf05-97a5c53c1c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:36.487672 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c59e76-697f-4d5f-bf05-97a5c53c1c86 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:36.487950 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c59e76-697f-4d5f-bf05-97a5c53c1c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:36.488231 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c59e76-697f-4d5f-bf05-97a5c53c1c86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:36.499235 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c59e76-697f-4d5f-bf05-97a5c53c1c86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:36.500706 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c59e76-697f-4d5f-bf05-97a5c53c1c86 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:36.514674 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-a1c59e76-697f-4d5f-bf05-97a5c53c1c86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:36.514674 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:36.514674 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:36.514674 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:36.514674 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:36.514674 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:36.514674 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:36.514674 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:36.514674 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:36.514674 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:36.514674 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:36.514674 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:36.514674 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:36.514674 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:36.514674 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:36.514674 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:36.516124 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:36.516124 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:36.516124 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:36.516124 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:36.516124 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:36.516124 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:36.516124 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:36.516124 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:36.516124 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:36.516124 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:36.516124 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:36.516124 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:36.516124 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:36.516124 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:36.516124 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:36.516124 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:36.516124 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:36.517270 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:36.517270 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:36.517270 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:36.517270 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 338/659] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:36.517270 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8936b8a3-e410-4949-ac74-f349dfae8115 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:36.517551 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8936b8a3-e410-4949-ac74-f349dfae8115 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:36.517710 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8936b8a3-e410-4949-ac74-f349dfae8115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:36.539500 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-8936b8a3-e410-4949-ac74-f349dfae8115 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:36.540657 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-8936b8a3-e410-4949-ac74-f349dfae8115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:36.578942 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:36.582636 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caddd393-2175-4632-80a3-108255c07d03 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:36.612085 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 339/660] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:24:36.613011 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8936b8a3-e410-4949-ac74-f349dfae8115 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:36.621464 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb8fd7-9746-4277-8529-12c2dfbbb4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:36.622100 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb8fd7-9746-4277-8529-12c2dfbbb4dc None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:36.622500 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb8fd7-9746-4277-8529-12c2dfbbb4dc None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:36.622793 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb8fd7-9746-4277-8529-12c2dfbbb4dc None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:36.631995 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fb8fd7-9746-4277-8529-12c2dfbbb4dc None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:36.633492 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fb8fd7-9746-4277-8529-12c2dfbbb4dc None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:36.647505 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-33fb8fd7-9746-4277-8529-12c2dfbbb4dc None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 05:24:36.647505 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:36.647505 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:36.647505 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:36.647505 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:36.647505 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:36.647505 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:36.647505 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:36.647505 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:36.647505 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:36.647505 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:36.647505 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:36.647505 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:24:36.647505 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:24:36.647505 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:24:36.647505 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:24:36.648559 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:36.648559 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:36.648559 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:24:36.648559 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:36.648559 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:24:36.648559 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:36.648559 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:36.648559 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:36.648559 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:36.648559 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:36.648559 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:36.648559 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:36.648559 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:36.648559 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:36.648559 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:36.648559 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:36.648559 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:36.648559 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:36.649517 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:36.649517 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:36.649517 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:24:36.649517 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:24:36.649517 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:24:36.649517 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:24:36.649517 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:24:36.649517 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:24:36.649517 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 05:24:36.649517 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:36.649517 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 322/661] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Aug 24 05:24:36 2022] GET /identity/v3/users/neutron => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:36.649517 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb8fd7-9746-4277-8529-12c2dfbbb4dc None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:36.674110 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:36.742987 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8936b8a3-e410-4949-ac74-f349dfae8115 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:36.765407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb8fd7-9746-4277-8529-12c2dfbbb4dc None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:36.780622 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3666cb-86c1-4ea2-8c40-5fce3ee6a71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:36.781510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3666cb-86c1-4ea2-8c40-5fce3ee6a71a None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:36.781805 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3666cb-86c1-4ea2-8c40-5fce3ee6a71a None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:36.782054 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3666cb-86c1-4ea2-8c40-5fce3ee6a71a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:36.789316 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3666cb-86c1-4ea2-8c40-5fce3ee6a71a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:36.790932 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3666cb-86c1-4ea2-8c40-5fce3ee6a71a None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:36.801386 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0708bb7e-880b-4f5c-a2f2-201f4ecee27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:36.802091 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708bb7e-880b-4f5c-a2f2-201f4ecee27c None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:36.802435 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708bb7e-880b-4f5c-a2f2-201f4ecee27c None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:36.802771 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708bb7e-880b-4f5c-a2f2-201f4ecee27c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:36.803347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-8d3666cb-86c1-4ea2-8c40-5fce3ee6a71a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:36.803347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:36.803347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:36.803347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:36.803347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:36.803347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:36.803347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:36.803347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:36.803347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:36.803347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:36.803347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:36.803347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:36.803347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:24:36.803347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:24:36.803347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:24:36.804363 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:24:36.804363 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:36.804363 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:36.804363 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:36.804363 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:36.804363 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:36.804363 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:36.804363 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:36.804363 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:36.804363 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:36.804363 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:36.804363 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:36.804363 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:36.804363 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:36.804363 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:36.804363 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:36.804363 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:36.805296 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:24:36.805296 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:24:36.805296 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:24:36.805296 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:24:36.805296 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:24:36.805296 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:24:36.805296 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:36.805296 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:36.805296 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 340/662] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:24:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:36.805296 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3666cb-86c1-4ea2-8c40-5fce3ee6a71a None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:36.814881 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0708bb7e-880b-4f5c-a2f2-201f4ecee27c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:36.816721 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0708bb7e-880b-4f5c-a2f2-201f4ecee27c None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:36.828741 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-0708bb7e-880b-4f5c-a2f2-201f4ecee27c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:36.828741 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:36.828741 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:36.828741 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:36.828741 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:36.828741 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:36.828741 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:36.828741 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:36.828741 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:36.828741 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:36.828741 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:36.828741 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:36.828741 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:24:36.828741 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:24:36.828741 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:24:36.830101 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:24:36.830101 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:36.830101 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:36.830101 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:36.830101 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:36.830101 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:36.830101 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:36.830101 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:36.830101 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:36.830101 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:36.830101 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:36.830101 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:36.830101 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:36.830101 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:36.830101 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:36.830101 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:36.830101 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:36.830992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:24:36.830992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:24:36.830992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:24:36.830992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:24:36.830992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:24:36.830992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:24:36.830992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:36.830992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:36.830992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 323/663] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:24:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:36.831479 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0708bb7e-880b-4f5c-a2f2-201f4ecee27c None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:36.854870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:36.923535 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3666cb-86c1-4ea2-8c40-5fce3ee6a71a None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:36.925957 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:36.940087 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0708bb7e-880b-4f5c-a2f2-201f4ecee27c None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:36.957692 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a431836e-77ad-43e5-9af8-9336af7fd76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:36.958455 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a431836e-77ad-43e5-9af8-9336af7fd76f None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:36.958866 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a431836e-77ad-43e5-9af8-9336af7fd76f None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:36.959147 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a431836e-77ad-43e5-9af8-9336af7fd76f None admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:36.959787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a431836e-77ad-43e5-9af8-9336af7fd76f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:36.960096 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a431836e-77ad-43e5-9af8-9336af7fd76f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:36.961501 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a431836e-77ad-43e5-9af8-9336af7fd76f None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:36.973506 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 341/664] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:36.974333 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a431836e-77ad-43e5-9af8-9336af7fd76f None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:36.979507 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c083267c-38a1-4837-ab73-ff223b919d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:36.980178 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c083267c-38a1-4837-ab73-ff223b919d92 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:36.980505 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c083267c-38a1-4837-ab73-ff223b919d92 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:36.980812 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c083267c-38a1-4837-ab73-ff223b919d92 None admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:36.981654 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c083267c-38a1-4837-ab73-ff223b919d92 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:36.981966 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c083267c-38a1-4837-ab73-ff223b919d92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:36.983961 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c083267c-38a1-4837-ab73-ff223b919d92 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:37.003508 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 324/665] 10.222.0.15 () {56 vars in 1139 bytes} [Wed Aug 24 05:24:36 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:37.004503 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c083267c-38a1-4837-ab73-ff223b919d92 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:37.079868 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a431836e-77ad-43e5-9af8-9336af7fd76f None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:37.084237 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:37.106899 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c083267c-38a1-4837-ab73-ff223b919d92 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:37.115445 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d787016-383e-4349-a1ce-504076de055a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:37.116068 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d787016-383e-4349-a1ce-504076de055a None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:37.116350 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d787016-383e-4349-a1ce-504076de055a None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:37.116659 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d787016-383e-4349-a1ce-504076de055a None admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:37.117351 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d787016-383e-4349-a1ce-504076de055a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:37.117661 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d787016-383e-4349-a1ce-504076de055a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:37.119098 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d787016-383e-4349-a1ce-504076de055a None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:37.132310 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 342/666] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Aug 24 05:24:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:37.133151 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d787016-383e-4349-a1ce-504076de055a None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:37.142553 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b241a-cdf9-4246-9429-ab4e69ce531b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:37.143162 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b241a-cdf9-4246-9429-ab4e69ce531b None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:37.143443 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b241a-cdf9-4246-9429-ab4e69ce531b None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:37.143720 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b241a-cdf9-4246-9429-ab4e69ce531b None admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:37.144432 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b241a-cdf9-4246-9429-ab4e69ce531b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:37.144740 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b241a-cdf9-4246-9429-ab4e69ce531b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:37.146158 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b241a-cdf9-4246-9429-ab4e69ce531b None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:37.158526 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:37.158736 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 325/667] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Aug 24 05:24:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:37.159509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b241a-cdf9-4246-9429-ab4e69ce531b None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:37.240385 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d787016-383e-4349-a1ce-504076de055a None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:37.263483 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b241a-cdf9-4246-9429-ab4e69ce531b None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:37.276558 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f331e348-90bd-4283-bd86-a3ea606823b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:37.277194 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331e348-90bd-4283-bd86-a3ea606823b2 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:37.277476 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331e348-90bd-4283-bd86-a3ea606823b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:37.277757 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331e348-90bd-4283-bd86-a3ea606823b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:37.288105 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f331e348-90bd-4283-bd86-a3ea606823b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:24:37.292880 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-f331e348-90bd-4283-bd86-a3ea606823b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:24:37.292880 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:37.292880 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:37.292880 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:37.292880 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:37.292880 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:37.292880 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:37.292880 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:37.292880 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:37.292880 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:37.292880 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:37.292880 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:37.292880 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:24:37.292880 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:24:37.292880 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:24:37.292880 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:24:37.293831 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:24:37.293831 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:24:37.293831 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:24:37.293831 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:24:37.293831 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:37.293831 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:37.293831 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:37.293831 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:37.293831 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:37.293831 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:37.293831 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:37.293831 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:37.293831 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:37.293831 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:37.293831 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:37.293831 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:37.294647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:37.294647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:37.294647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:37.294647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:37.294647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:24:37.294647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:24:37.294647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:24:37.294647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:24:37.294647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:24:37.294647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:24:37.294647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:24:37.294647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:37.294647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 343/668] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Aug 24 05:24:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:37.296258 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f331e348-90bd-4283-bd86-a3ea606823b2 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:37.297122 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5e852-b6a7-4cca-963a-9919b63aedd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:37.297814 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5e852-b6a7-4cca-963a-9919b63aedd2 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:37.298090 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5e852-b6a7-4cca-963a-9919b63aedd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:37.298360 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5e852-b6a7-4cca-963a-9919b63aedd2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:37.301578 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:37.305907 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d5e852-b6a7-4cca-963a-9919b63aedd2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:37.307378 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d5e852-b6a7-4cca-963a-9919b63aedd2 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:37.319804 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-73d5e852-b6a7-4cca-963a-9919b63aedd2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:37.319804 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:37.319804 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:37.319804 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:37.319804 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:37.319804 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:37.319804 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:37.319804 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:37.319804 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:37.319804 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:37.319804 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:37.319804 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:37.319804 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:24:37.319804 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:24:37.319804 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:24:37.320997 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:24:37.320997 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:37.320997 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:37.320997 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:37.320997 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:37.320997 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:37.320997 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:37.320997 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:37.320997 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:37.320997 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:37.320997 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:37.320997 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:37.320997 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:37.320997 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:37.320997 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:37.320997 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:37.320997 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:37.321934 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:24:37.321934 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:24:37.321934 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:24:37.321934 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:24:37.321934 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:24:37.321934 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:24:37.321934 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:37.321934 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:37.321934 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 326/669] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:24:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:37.321934 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5e852-b6a7-4cca-963a-9919b63aedd2 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:37.388100 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:37.403648 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f331e348-90bd-4283-bd86-a3ea606823b2 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:37.425665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5e852-b6a7-4cca-963a-9919b63aedd2 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:37.441032 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0242f-4473-41ac-a94e-dc3f6ad2dc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:37.441791 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0242f-4473-41ac-a94e-dc3f6ad2dc6a None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:37.442070 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0242f-4473-41ac-a94e-dc3f6ad2dc6a None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:37.442627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0242f-4473-41ac-a94e-dc3f6ad2dc6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:37.459532 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89813c88-2b4a-489f-96f6-2d3464683669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:37.460133 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89813c88-2b4a-489f-96f6-2d3464683669 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:37.460432 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89813c88-2b4a-489f-96f6-2d3464683669 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:37.460719 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89813c88-2b4a-489f-96f6-2d3464683669 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:37.461446 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89813c88-2b4a-489f-96f6-2d3464683669 None admin] RBAC: Adding query filter params (role.id=cb40cd8499f7418ca0da5ea0f1b87ff9, scope.project.id=59592706ae9442e08353158c625e9581, user.id=fbecffb800584c5786813accd061421b) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:37.461757 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89813c88-2b4a-489f-96f6-2d3464683669 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:37.463203 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89813c88-2b4a-489f-96f6-2d3464683669 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:37.469923 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 327/670] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:24:37 2022] GET /identity/v3/role_assignments?user.id=fbecffb800584c5786813accd061421b&scope.project.id=59592706ae9442e08353158c625e9581&role.id=cb40cd8499f7418ca0da5ea0f1b87ff9 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:37.470755 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89813c88-2b4a-489f-96f6-2d3464683669 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:37.520472 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c0242f-4473-41ac-a94e-dc3f6ad2dc6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:37.521993 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c0242f-4473-41ac-a94e-dc3f6ad2dc6a None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:37.542730 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:37.587338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89813c88-2b4a-489f-96f6-2d3464683669 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:37.627016 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed998b-7077-4d7a-a465-e291a2264e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:37.627795 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed998b-7077-4d7a-a465-e291a2264e82 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:37.628080 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed998b-7077-4d7a-a465-e291a2264e82 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:37.628353 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed998b-7077-4d7a-a465-e291a2264e82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:37.629213 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ed998b-7077-4d7a-a465-e291a2264e82 None admin] RBAC: Adding query filter params (role.id=cb40cd8499f7418ca0da5ea0f1b87ff9, scope.project.id=59592706ae9442e08353158c625e9581, user.id=9aee9c524a974a248c3bf363444f584a) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:37.629522 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ed998b-7077-4d7a-a465-e291a2264e82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:37.630064 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:37.630964 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ed998b-7077-4d7a-a465-e291a2264e82 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:37.640047 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 328/671] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:24:37 2022] GET /identity/v3/role_assignments?user.id=9aee9c524a974a248c3bf363444f584a&scope.project.id=59592706ae9442e08353158c625e9581&role.id=cb40cd8499f7418ca0da5ea0f1b87ff9 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:37.640047 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed998b-7077-4d7a-a465-e291a2264e82 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:37.648203 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 344/672] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:24:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:24:37.649135 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0242f-4473-41ac-a94e-dc3f6ad2dc6a None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:37.773150 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed998b-7077-4d7a-a465-e291a2264e82 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:37.795114 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0242f-4473-41ac-a94e-dc3f6ad2dc6a None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:37.804247 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:37.817500 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89408ed4-9120-4959-9cc7-56cafd9515a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:37.818285 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89408ed4-9120-4959-9cc7-56cafd9515a1 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:37.818663 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89408ed4-9120-4959-9cc7-56cafd9515a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:37.819049 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89408ed4-9120-4959-9cc7-56cafd9515a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:37.819820 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89408ed4-9120-4959-9cc7-56cafd9515a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:37.820225 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89408ed4-9120-4959-9cc7-56cafd9515a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:37.821744 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89408ed4-9120-4959-9cc7-56cafd9515a1 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:37.832698 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 329/673] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Aug 24 05:24:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:37.833861 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89408ed4-9120-4959-9cc7-56cafd9515a1 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:37.836130 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e7935-e8dd-45a6-a567-fa029b2432df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:37.837062 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e7935-e8dd-45a6-a567-fa029b2432df None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:37.837409 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e7935-e8dd-45a6-a567-fa029b2432df None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:37.838680 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e7935-e8dd-45a6-a567-fa029b2432df None admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:37.839459 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e7935-e8dd-45a6-a567-fa029b2432df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:37.839834 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e7935-e8dd-45a6-a567-fa029b2432df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:37.841368 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e7935-e8dd-45a6-a567-fa029b2432df None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:37.857431 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 345/674] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Aug 24 05:24:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:37.860756 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e7935-e8dd-45a6-a567-fa029b2432df None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:37.923965 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:37.967283 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89408ed4-9120-4959-9cc7-56cafd9515a1 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:37.997545 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e7935-e8dd-45a6-a567-fa029b2432df None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:38.012647 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d420d-87ac-4219-8082-d12ba0d3c45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:38.013286 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d420d-87ac-4219-8082-d12ba0d3c45f None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:38.013867 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d420d-87ac-4219-8082-d12ba0d3c45f None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:38.014142 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d420d-87ac-4219-8082-d12ba0d3c45f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:38.027455 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906d420d-87ac-4219-8082-d12ba0d3c45f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:38.029222 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906d420d-87ac-4219-8082-d12ba0d3c45f None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:38.043627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2991826b-fb6f-46a8-95d5-62c32e71826c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:38.044640 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991826b-fb6f-46a8-95d5-62c32e71826c None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:38.045145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991826b-fb6f-46a8-95d5-62c32e71826c None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:38.046203 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991826b-fb6f-46a8-95d5-62c32e71826c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:38.047713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-906d420d-87ac-4219-8082-d12ba0d3c45f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:38.047713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:38.047713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:38.047713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:38.047713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:38.047713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:38.047713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:38.047713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:38.047713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:38.047713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:38.047713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:38.047713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:38.047713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:38.047713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:38.047713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:38.047713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:38.048638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:38.048638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:38.048638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:38.048638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:38.048638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:38.048638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:38.048638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:38.048638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:38.048638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:38.048638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:38.048638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:38.048638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:38.048638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:38.048638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:38.048638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:38.048638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:38.048638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:38.049554 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:38.049554 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:38.049554 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:38.052483 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 330/675] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:38.054216 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d420d-87ac-4219-8082-d12ba0d3c45f None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:38.065107 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2991826b-fb6f-46a8-95d5-62c32e71826c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:38.066624 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2991826b-fb6f-46a8-95d5-62c32e71826c None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:38.084538 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-2991826b-fb6f-46a8-95d5-62c32e71826c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:38.084538 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:38.084538 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:38.084538 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:38.084538 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:38.084538 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:38.084538 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:38.084538 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:38.084538 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:38.084538 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:38.084538 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:38.084538 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:38.084538 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:38.084538 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:38.084538 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:38.084538 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:38.085545 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:38.085545 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:38.085545 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:38.085545 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:38.085545 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:38.085545 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:38.085545 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:38.085545 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:38.085545 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:38.085545 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:38.085545 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:38.085545 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:38.085545 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:38.085545 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:38.085545 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:38.085545 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:38.085545 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:38.087082 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:38.087082 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:38.087082 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:38.087082 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 346/676] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:38.087082 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2991826b-fb6f-46a8-95d5-62c32e71826c None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:38.111080 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:38.182775 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d420d-87ac-4219-8082-d12ba0d3c45f None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:38.188663 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:38.194857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2991826b-fb6f-46a8-95d5-62c32e71826c None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:38.230414 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d550290e-e549-4ad3-b9df-760f60d8c745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:38.231502 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e593a40e-8c8f-4576-a3f5-87059a5bc40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:38.231734 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d550290e-e549-4ad3-b9df-760f60d8c745 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:38.232109 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593a40e-8c8f-4576-a3f5-87059a5bc40b None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:38.232392 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593a40e-8c8f-4576-a3f5-87059a5bc40b None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:38.232685 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d550290e-e549-4ad3-b9df-760f60d8c745 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:38.232785 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593a40e-8c8f-4576-a3f5-87059a5bc40b None admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:38.233835 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d550290e-e549-4ad3-b9df-760f60d8c745 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:38.234042 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e593a40e-8c8f-4576-a3f5-87059a5bc40b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:38.234354 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e593a40e-8c8f-4576-a3f5-87059a5bc40b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:38.235163 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d550290e-e549-4ad3-b9df-760f60d8c745 None admin] RBAC: Adding query filter params (role.id=cb40cd8499f7418ca0da5ea0f1b87ff9, scope.project.id=59592706ae9442e08353158c625e9581, user.id=47bfd618ef39498f91fe7731cd40253b) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:38.235673 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d550290e-e549-4ad3-b9df-760f60d8c745 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:38.235773 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e593a40e-8c8f-4576-a3f5-87059a5bc40b None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:38.237753 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d550290e-e549-4ad3-b9df-760f60d8c745 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:38.246285 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 331/677] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:24:38 2022] GET /identity/v3/role_assignments?user.id=47bfd618ef39498f91fe7731cd40253b&scope.project.id=59592706ae9442e08353158c625e9581&role.id=cb40cd8499f7418ca0da5ea0f1b87ff9 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:38.247502 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d550290e-e549-4ad3-b9df-760f60d8c745 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:38.259076 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 347/678] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:38.269364 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e593a40e-8c8f-4576-a3f5-87059a5bc40b None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:38.427696 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d550290e-e549-4ad3-b9df-760f60d8c745 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:38.429810 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:38.457856 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e593a40e-8c8f-4576-a3f5-87059a5bc40b None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:38.476335 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85834b05-9c18-4b08-ad21-01b1fbcc260e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:38.477211 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85834b05-9c18-4b08-ad21-01b1fbcc260e None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:38.477392 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85834b05-9c18-4b08-ad21-01b1fbcc260e None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:38.477627 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85834b05-9c18-4b08-ad21-01b1fbcc260e None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:38.478290 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85834b05-9c18-4b08-ad21-01b1fbcc260e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:38.478598 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85834b05-9c18-4b08-ad21-01b1fbcc260e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:38.480002 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85834b05-9c18-4b08-ad21-01b1fbcc260e None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:38.493146 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 332/679] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:38.499306 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85834b05-9c18-4b08-ad21-01b1fbcc260e None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:38.502512 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d569494-48ce-43e8-87e4-89f41f099011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:38.503359 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d569494-48ce-43e8-87e4-89f41f099011 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:38.503782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d569494-48ce-43e8-87e4-89f41f099011 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:38.504200 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d569494-48ce-43e8-87e4-89f41f099011 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:38.523050 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d569494-48ce-43e8-87e4-89f41f099011 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:38.525269 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d569494-48ce-43e8-87e4-89f41f099011 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:38.542305 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-9d569494-48ce-43e8-87e4-89f41f099011 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:38.542305 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:38.542305 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:38.542305 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:38.542305 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:38.542305 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:38.542305 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:38.542305 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:38.542305 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:38.542305 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:38.542305 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:38.542305 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:38.542305 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:24:38.542305 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:24:38.542305 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:24:38.543420 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:24:38.543420 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:38.543420 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:38.543420 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:38.543420 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:38.543420 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:38.543420 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:38.543420 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:38.543420 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:38.543420 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:38.543420 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:38.543420 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:38.543420 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:38.543420 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:38.543420 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:38.543420 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:38.543420 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:38.544516 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:24:38.544516 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:24:38.544516 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:24:38.544516 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:24:38.544516 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:24:38.544516 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:24:38.544516 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:38.544516 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:38.551533 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 348/680] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:24:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:38.553295 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d569494-48ce-43e8-87e4-89f41f099011 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:38.579284 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:38.643524 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85834b05-9c18-4b08-ad21-01b1fbcc260e None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:38.682136 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d569494-48ce-43e8-87e4-89f41f099011 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:38.690110 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3fd968-1352-4d58-a4e6-0a823bfd5d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:38.691418 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3fd968-1352-4d58-a4e6-0a823bfd5d51 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:38.691848 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3fd968-1352-4d58-a4e6-0a823bfd5d51 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:38.692267 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3fd968-1352-4d58-a4e6-0a823bfd5d51 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:38.703024 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3fd968-1352-4d58-a4e6-0a823bfd5d51 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:38.704713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3fd968-1352-4d58-a4e6-0a823bfd5d51 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:38.719948 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-de3fd968-1352-4d58-a4e6-0a823bfd5d51 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 05:24:38.719948 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:38.719948 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:38.719948 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:38.719948 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:38.719948 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:38.719948 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:38.719948 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:38.719948 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:38.719948 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:38.719948 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:38.719948 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:38.719948 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:24:38.719948 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:24:38.719948 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:24:38.719948 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:24:38.721117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:38.721117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:38.721117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:24:38.721117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:38.721117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:24:38.721117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:38.721117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:38.721117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:38.721117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:38.721117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:38.721117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:38.721117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:38.721117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:38.721117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:38.721117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:38.721117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:38.721117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:38.721117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:38.722368 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:38.722368 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:38.722368 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:24:38.722368 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:24:38.722368 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:24:38.722368 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:24:38.722368 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:24:38.722368 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:24:38.722368 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 05:24:38.722368 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:38.723037 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e66b8-495a-444c-aafd-03c9b28c18cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:38.723695 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e66b8-495a-444c-aafd-03c9b28c18cd None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:38.724068 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e66b8-495a-444c-aafd-03c9b28c18cd None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:38.724485 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e66b8-495a-444c-aafd-03c9b28c18cd None admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:38.725303 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391e66b8-495a-444c-aafd-03c9b28c18cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:38.725455 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 333/681] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Aug 24 05:24:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:24:38.729301 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391e66b8-495a-444c-aafd-03c9b28c18cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:38.729301 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391e66b8-495a-444c-aafd-03c9b28c18cd None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:38.731798 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3fd968-1352-4d58-a4e6-0a823bfd5d51 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:38.753220 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 349/682] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Aug 24 05:24:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:38.768912 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:38.774552 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e66b8-495a-444c-aafd-03c9b28c18cd None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:38.859263 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3fd968-1352-4d58-a4e6-0a823bfd5d51 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:38.898636 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f44155a-8f23-4312-ba23-d8693b650591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:38.899589 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44155a-8f23-4312-ba23-d8693b650591 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:38.900084 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44155a-8f23-4312-ba23-d8693b650591 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:38.900589 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44155a-8f23-4312-ba23-d8693b650591 None admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:38.901514 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f44155a-8f23-4312-ba23-d8693b650591 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:38.902031 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f44155a-8f23-4312-ba23-d8693b650591 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:38.903675 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f44155a-8f23-4312-ba23-d8693b650591 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:38.923412 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 334/683] 10.222.0.15 () {56 vars in 1135 bytes} [Wed Aug 24 05:24:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:38.928871 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f44155a-8f23-4312-ba23-d8693b650591 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:38.963488 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:39.010399 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e66b8-495a-444c-aafd-03c9b28c18cd None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:39.085833 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7a823e-cb87-41dc-a185-814e52a3baca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:39.086448 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a823e-cb87-41dc-a185-814e52a3baca None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:39.086738 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a823e-cb87-41dc-a185-814e52a3baca None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:39.087010 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a823e-cb87-41dc-a185-814e52a3baca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:39.087741 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7a823e-cb87-41dc-a185-814e52a3baca None admin] RBAC: Adding query filter params (role.id=cb40cd8499f7418ca0da5ea0f1b87ff9, scope.project.id=59592706ae9442e08353158c625e9581, user.id=b1ecd9b302b64ecf81d7e3d3356ddf7d) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:39.088051 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7a823e-cb87-41dc-a185-814e52a3baca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:39.089520 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7a823e-cb87-41dc-a185-814e52a3baca None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:39.104232 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 350/684] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:24:38 2022] GET /identity/v3/role_assignments?user.id=b1ecd9b302b64ecf81d7e3d3356ddf7d&scope.project.id=59592706ae9442e08353158c625e9581&role.id=cb40cd8499f7418ca0da5ea0f1b87ff9 => generated 608 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:39.111803 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:39.132668 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f44155a-8f23-4312-ba23-d8693b650591 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:39.171974 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a4780-5767-4280-9716-e53573842a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:39.172619 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a4780-5767-4280-9716-e53573842a40 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:39.172902 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a4780-5767-4280-9716-e53573842a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:39.173244 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a4780-5767-4280-9716-e53573842a40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:39.268178 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611a4780-5767-4280-9716-e53573842a40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:39.269705 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611a4780-5767-4280-9716-e53573842a40 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:39.376749 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:39.396263 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 335/685] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:24:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:24:39.401737 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7a823e-cb87-41dc-a185-814e52a3baca None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:39.523078 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7a823e-cb87-41dc-a185-814e52a3baca None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:39.566063 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea871652-097a-4de5-aff6-ed27cb1880e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:39.567141 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea871652-097a-4de5-aff6-ed27cb1880e3 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:39.567626 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea871652-097a-4de5-aff6-ed27cb1880e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:39.568151 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea871652-097a-4de5-aff6-ed27cb1880e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:39.581686 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:39.585058 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea871652-097a-4de5-aff6-ed27cb1880e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:39.586749 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea871652-097a-4de5-aff6-ed27cb1880e3 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:39.606844 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-ea871652-097a-4de5-aff6-ed27cb1880e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:39.606844 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:39.606844 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:39.606844 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:39.606844 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:39.606844 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:39.606844 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:39.606844 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:39.606844 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:39.606844 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:39.606844 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:39.606844 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:39.606844 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:39.606844 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:39.606844 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:39.606844 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:39.607843 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:39.607843 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:39.607843 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:39.607843 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:39.607843 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:39.607843 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:39.607843 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:39.607843 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:39.607843 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:39.607843 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:39.607843 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:39.607843 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:39.607843 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:39.607843 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:39.607843 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:39.607843 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:39.607843 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:39.609304 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:39.609304 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:39.609304 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:39.611568 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 351/686] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:39.617061 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a4780-5767-4280-9716-e53573842a40 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:39.763953 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a4780-5767-4280-9716-e53573842a40 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:39.815181 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3cdfcb-f1bd-458d-959a-6ec7a59ea984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:39.815719 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3cdfcb-f1bd-458d-959a-6ec7a59ea984 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:39.816005 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3cdfcb-f1bd-458d-959a-6ec7a59ea984 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:39.816286 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3cdfcb-f1bd-458d-959a-6ec7a59ea984 None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:39.816966 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3cdfcb-f1bd-458d-959a-6ec7a59ea984 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:39.817397 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3cdfcb-f1bd-458d-959a-6ec7a59ea984 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:39.818772 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3cdfcb-f1bd-458d-959a-6ec7a59ea984 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:39.836094 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 336/687] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:39.841789 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea871652-097a-4de5-aff6-ed27cb1880e3 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:39.863429 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f56b062-378d-4010-97ea-6b33558ee879 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:39.863738 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f56b062-378d-4010-97ea-6b33558ee879 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:39.863987 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f56b062-378d-4010-97ea-6b33558ee879 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:39.865677 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 337/688] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:24:39.870904 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573da2e9-8d0f-40fb-927c-73c4511115ff None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:39.871211 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573da2e9-8d0f-40fb-927c-73c4511115ff None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:39.871452 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573da2e9-8d0f-40fb-927c-73c4511115ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:39.907761 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-573da2e9-8d0f-40fb-927c-73c4511115ff None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:39.909007 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-573da2e9-8d0f-40fb-927c-73c4511115ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:39.953111 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:39.997318 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 338/689] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:24:39.999902 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae357e7-1f93-4d60-8732-be04faad1368 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:40.000470 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae357e7-1f93-4d60-8732-be04faad1368 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:40.001652 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae357e7-1f93-4d60-8732-be04faad1368 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:40.003956 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 339/690] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:40.010265 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e689e-82c0-483e-8a89-25be2c3402ea None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:40.010265 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e689e-82c0-483e-8a89-25be2c3402ea None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:40.010265 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e689e-82c0-483e-8a89-25be2c3402ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:40.041488 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-502e689e-82c0-483e-8a89-25be2c3402ea None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:40.042162 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea871652-097a-4de5-aff6-ed27cb1880e3 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:40.042662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-502e689e-82c0-483e-8a89-25be2c3402ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:40.079895 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208601bf-eb9b-4baa-9b35-8b5077f5bb59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:40.080486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208601bf-eb9b-4baa-9b35-8b5077f5bb59 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:40.080770 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208601bf-eb9b-4baa-9b35-8b5077f5bb59 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:40.081045 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208601bf-eb9b-4baa-9b35-8b5077f5bb59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:40.088093 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208601bf-eb9b-4baa-9b35-8b5077f5bb59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:40.089788 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208601bf-eb9b-4baa-9b35-8b5077f5bb59 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:40.101741 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-208601bf-eb9b-4baa-9b35-8b5077f5bb59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:40.101741 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:40.101741 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:40.101741 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:40.101741 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:40.101741 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:40.101741 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:40.101741 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:40.101741 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:40.101741 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:40.101741 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:40.101741 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:40.101741 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:24:40.101741 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:24:40.101741 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:24:40.102644 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:24:40.102644 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:40.102644 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:40.102644 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:40.102644 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:40.102644 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:40.102644 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:40.102644 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:40.102644 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:40.102644 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:40.102644 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:40.102644 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:40.102644 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:40.102644 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:40.102644 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:40.102644 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:40.102644 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:40.103510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:24:40.103510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:24:40.103510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:24:40.103510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:24:40.103510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:24:40.103510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:24:40.103510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:40.103510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:40.103510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 352/691] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:24:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:40.109737 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208601bf-eb9b-4baa-9b35-8b5077f5bb59 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:40.171551 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:40.235855 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208601bf-eb9b-4baa-9b35-8b5077f5bb59 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:40.262074 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 340/692] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:24:40.282081 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14dba9e-a960-47f8-855a-74474b3ba90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:40.282701 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14dba9e-a960-47f8-855a-74474b3ba90e None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:40.282980 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14dba9e-a960-47f8-855a-74474b3ba90e None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:40.283254 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14dba9e-a960-47f8-855a-74474b3ba90e None admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:40.283946 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14dba9e-a960-47f8-855a-74474b3ba90e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:40.284251 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14dba9e-a960-47f8-855a-74474b3ba90e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:40.285710 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14dba9e-a960-47f8-855a-74474b3ba90e None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:40.303377 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 353/693] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Aug 24 05:24:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:40.310267 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e689e-82c0-483e-8a89-25be2c3402ea None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:40.375398 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14dba9e-a960-47f8-855a-74474b3ba90e None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:40.496585 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:40.503077 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e689e-82c0-483e-8a89-25be2c3402ea None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:40.553690 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14dba9e-a960-47f8-855a-74474b3ba90e None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:40.560883 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060bd54-1b48-4b5e-b622-5b4508d08523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:40.561549 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060bd54-1b48-4b5e-b622-5b4508d08523 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:40.561831 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060bd54-1b48-4b5e-b622-5b4508d08523 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:40.562107 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060bd54-1b48-4b5e-b622-5b4508d08523 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:40.562871 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9060bd54-1b48-4b5e-b622-5b4508d08523 None admin] RBAC: Adding query filter params (role.id=cb40cd8499f7418ca0da5ea0f1b87ff9, scope.project.id=59592706ae9442e08353158c625e9581, user.id=bb2b6cae4a004539a95fc7dd558db255) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:40.563214 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9060bd54-1b48-4b5e-b622-5b4508d08523 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:40.564855 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9060bd54-1b48-4b5e-b622-5b4508d08523 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:40.573006 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:40.579911 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 341/694] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:24:40 2022] GET /identity/v3/role_assignments?user.id=bb2b6cae4a004539a95fc7dd558db255&scope.project.id=59592706ae9442e08353158c625e9581&role.id=cb40cd8499f7418ca0da5ea0f1b87ff9 => generated 608 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:40.595421 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ae5b3-df23-4d5d-b4bf-d20165d5e76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:40.596481 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ae5b3-df23-4d5d-b4bf-d20165d5e76a None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:40.596972 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ae5b3-df23-4d5d-b4bf-d20165d5e76a None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:40.597450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ae5b3-df23-4d5d-b4bf-d20165d5e76a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:40.610268 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770ae5b3-df23-4d5d-b4bf-d20165d5e76a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:24:40.614651 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060bd54-1b48-4b5e-b622-5b4508d08523 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:40.616043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-770ae5b3-df23-4d5d-b4bf-d20165d5e76a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:24:40.616043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:40.616043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:40.616043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:40.616043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:40.616043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:40.616043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:40.616043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:40.616043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:40.616043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:40.616043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:40.616043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:40.616043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:24:40.616043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:24:40.616043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:24:40.616043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:24:40.617297 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:24:40.617297 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:24:40.617297 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:24:40.617297 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:24:40.617297 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:40.617297 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:40.617297 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:40.617297 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:40.617297 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:40.617297 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:40.617297 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:40.617297 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:40.617297 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:40.617297 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:40.617297 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:40.617297 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:40.618156 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:40.618156 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:40.618156 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:40.618156 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:40.618156 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:24:40.618156 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:24:40.618156 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:24:40.618156 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:24:40.618156 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:24:40.618156 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:24:40.618156 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:24:40.618156 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:40.620906 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 354/695] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Aug 24 05:24:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:24:40.627114 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ae5b3-df23-4d5d-b4bf-d20165d5e76a None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:40.784637 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060bd54-1b48-4b5e-b622-5b4508d08523 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:40.830593 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee46f5-8643-444a-adbd-ff754c89415b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:40.831888 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee46f5-8643-444a-adbd-ff754c89415b None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:40.831888 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee46f5-8643-444a-adbd-ff754c89415b None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:40.831888 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee46f5-8643-444a-adbd-ff754c89415b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:40.843661 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eee46f5-8643-444a-adbd-ff754c89415b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:40.845161 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eee46f5-8643-444a-adbd-ff754c89415b None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:40.862806 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-7eee46f5-8643-444a-adbd-ff754c89415b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:40.862806 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:40.862806 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:40.862806 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:40.862806 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:40.862806 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:40.862806 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:40.862806 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:40.862806 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:40.862806 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:40.862806 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:40.862806 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:40.862806 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:40.862806 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:40.862806 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:40.862806 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:40.863801 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:40.863801 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:40.863801 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:40.863801 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:40.863801 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:40.863801 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:40.863801 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:40.863801 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:40.863801 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:40.863801 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:40.863801 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:40.863801 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:40.863801 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:40.863801 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:40.863801 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:40.863801 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:40.863801 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:40.864697 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:40.864697 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:40.864697 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:40.866608 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 342/696] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:24:40.873520 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee46f5-8643-444a-adbd-ff754c89415b None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:40.913481 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ae5b3-df23-4d5d-b4bf-d20165d5e76a None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:40.915553 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:40.957174 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b9557d-e75f-42e5-bb7f-6f72a97c9dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:40.957854 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9557d-e75f-42e5-bb7f-6f72a97c9dd4 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:40.958220 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9557d-e75f-42e5-bb7f-6f72a97c9dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:40.958498 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9557d-e75f-42e5-bb7f-6f72a97c9dd4 None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:40.959367 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b9557d-e75f-42e5-bb7f-6f72a97c9dd4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:40.959674 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b9557d-e75f-42e5-bb7f-6f72a97c9dd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:40.961084 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b9557d-e75f-42e5-bb7f-6f72a97c9dd4 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:40.972792 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 355/697] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:24:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:24:40.979348 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b9557d-e75f-42e5-bb7f-6f72a97c9dd4 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:41.016232 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee46f5-8643-444a-adbd-ff754c89415b None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:41.027674 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:41.059296 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f3ea6-9d7f-4ee3-a3e8-f9f313b1f198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:41.059903 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f3ea6-9d7f-4ee3-a3e8-f9f313b1f198 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:41.060241 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f3ea6-9d7f-4ee3-a3e8-f9f313b1f198 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:41.060557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f3ea6-9d7f-4ee3-a3e8-f9f313b1f198 None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:41.062216 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f3ea6-9d7f-4ee3-a3e8-f9f313b1f198 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:41.062525 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f3ea6-9d7f-4ee3-a3e8-f9f313b1f198 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:41.063887 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f3ea6-9d7f-4ee3-a3e8-f9f313b1f198 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:41.081156 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 343/698] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:41.082992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406630c5-a8ae-4356-8d42-c3af2fc9596b None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:41.083494 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406630c5-a8ae-4356-8d42-c3af2fc9596b None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:41.083995 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406630c5-a8ae-4356-8d42-c3af2fc9596b None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:41.085500 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 344/699] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:41.088245 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406630c5-a8ae-4356-8d42-c3af2fc9596b None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:41.106032 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b9557d-e75f-42e5-bb7f-6f72a97c9dd4 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:41.142948 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7376d5c1-f9c3-47f9-9467-9619871231c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:41.143753 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376d5c1-f9c3-47f9-9467-9619871231c2 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:41.144095 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376d5c1-f9c3-47f9-9467-9619871231c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:41.144479 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376d5c1-f9c3-47f9-9467-9619871231c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:41.155824 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7376d5c1-f9c3-47f9-9467-9619871231c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:41.157626 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7376d5c1-f9c3-47f9-9467-9619871231c2 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:41.173882 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-7376d5c1-f9c3-47f9-9467-9619871231c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:41.173882 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:41.173882 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:41.173882 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:41.173882 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:41.173882 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:41.173882 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:41.173882 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:41.173882 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:41.173882 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:41.173882 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:41.173882 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:41.173882 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:41.173882 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:41.173882 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:41.173882 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:41.174867 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:41.174867 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:41.174867 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:41.174867 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:41.174867 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:41.174867 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:41.174867 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:41.174867 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:41.174867 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:41.174867 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:41.174867 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:41.174867 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:41.174867 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:41.174867 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:41.174867 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:41.174867 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:41.174867 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:41.175786 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:41.175786 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:41.175786 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:41.175786 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 356/700] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:41.176748 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a0d220-970a-4973-ad9a-43076bef0630 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:41.177028 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a0d220-970a-4973-ad9a-43076bef0630 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:41.177305 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a0d220-970a-4973-ad9a-43076bef0630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:41.202561 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-99a0d220-970a-4973-ad9a-43076bef0630 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:41.203072 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:41.204137 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-99a0d220-970a-4973-ad9a-43076bef0630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:41.239432 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406630c5-a8ae-4356-8d42-c3af2fc9596b None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:41.278274 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17353b68-96cf-435c-8ce8-ad4dd1c0d4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:41.278993 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17353b68-96cf-435c-8ce8-ad4dd1c0d4b6 None admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:41.279329 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17353b68-96cf-435c-8ce8-ad4dd1c0d4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:41.279737 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17353b68-96cf-435c-8ce8-ad4dd1c0d4b6 None admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:41.281288 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17353b68-96cf-435c-8ce8-ad4dd1c0d4b6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:41.282836 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17353b68-96cf-435c-8ce8-ad4dd1c0d4b6 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:41.291500 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 357/701] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:24:41.293215 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a0d220-970a-4973-ad9a-43076bef0630 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:41.319240 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:41.405303 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a0d220-970a-4973-ad9a-43076bef0630 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:41.433786 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 345/702] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Aug 24 05:24:41 2022] POST /identity/v3/users => generated 254 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:24:41.450249 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb249983-169b-48ef-8e89-afb6632bc902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:41.450881 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb249983-169b-48ef-8e89-afb6632bc902 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:41.451158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb249983-169b-48ef-8e89-afb6632bc902 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:41.451507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb249983-169b-48ef-8e89-afb6632bc902 None admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:41.452601 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb249983-169b-48ef-8e89-afb6632bc902 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:41.452904 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb249983-169b-48ef-8e89-afb6632bc902 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:41.454375 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb249983-169b-48ef-8e89-afb6632bc902 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:41.468184 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 358/703] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:41.473962 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17353b68-96cf-435c-8ce8-ad4dd1c0d4b6 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:41.564775 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf5af0-3e00-457d-9998-ee9d589c72c0 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:41.565069 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf5af0-3e00-457d-9998-ee9d589c72c0 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:41.565352 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf5af0-3e00-457d-9998-ee9d589c72c0 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:41.568695 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 359/704] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:41.576513 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdf5af0-3e00-457d-9998-ee9d589c72c0 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:41.634845 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17353b68-96cf-435c-8ce8-ad4dd1c0d4b6 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:41.683290 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:41.694002 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cf28dd-de17-4370-a81c-62385cb879f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:41.694002 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf28dd-de17-4370-a81c-62385cb879f6 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:41.694002 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf28dd-de17-4370-a81c-62385cb879f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:41.694002 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf28dd-de17-4370-a81c-62385cb879f6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:41.697886 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:41.701167 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cf28dd-de17-4370-a81c-62385cb879f6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:41.704676 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cf28dd-de17-4370-a81c-62385cb879f6 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:41.716190 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-39cf28dd-de17-4370-a81c-62385cb879f6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:24:41.716190 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:41.716190 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:41.716190 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:41.716190 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:41.716190 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:41.716190 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:41.716190 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:41.716190 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:41.716190 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:41.716190 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:41.716190 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:41.716190 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:24:41.716190 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:24:41.716190 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:24:41.716190 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:24:41.717890 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:41.717890 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:41.717890 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:24:41.717890 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:41.717890 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:24:41.717890 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:41.717890 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:41.717890 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:41.717890 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:41.717890 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:41.717890 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:41.717890 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:41.717890 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:41.717890 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:41.717890 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:41.717890 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:41.717890 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:41.717890 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:41.718839 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:41.718839 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:41.718839 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:24:41.718839 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:24:41.718839 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:24:41.718839 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:24:41.718839 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:24:41.718839 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:24:41.718839 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:24:41.718839 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:41.719366 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 346/705] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Aug 24 05:24:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:41.720844 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d74ce3-75f7-4ea8-8e8e-7b947c331049 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:41.721212 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d74ce3-75f7-4ea8-8e8e-7b947c331049 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:41.721491 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d74ce3-75f7-4ea8-8e8e-7b947c331049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:41.739239 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdf5af0-3e00-457d-9998-ee9d589c72c0 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:41.743718 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-00d74ce3-75f7-4ea8-8e8e-7b947c331049 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:41.745049 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-00d74ce3-75f7-4ea8-8e8e-7b947c331049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:41.789869 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62c6eb-8266-4677-af19-7aae60d58715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:41.790529 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62c6eb-8266-4677-af19-7aae60d58715 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:41.790812 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62c6eb-8266-4677-af19-7aae60d58715 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:41.791094 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62c6eb-8266-4677-af19-7aae60d58715 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:41.791688 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62c6eb-8266-4677-af19-7aae60d58715 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:41.793121 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62c6eb-8266-4677-af19-7aae60d58715 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:41.810700 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-4e62c6eb-8266-4677-af19-7aae60d58715 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 05:24:41.810700 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:41.810700 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:24:41.810700 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:24:41.810700 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:24:41.810700 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:24:41.810700 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:24:41.810700 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:24:41.810700 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 05:24:41.810700 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:41.810700 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:24:41.810700 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:41.810700 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:41.810700 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:41.810700 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:41.810700 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:41.810700 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:41.812756 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:41.812756 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:41.812756 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:41.812756 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:41.812756 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:41.812756 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:41.812756 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:24:41.812756 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:24:41.812756 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:24:41.812756 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:24:41.812756 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:41.812756 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:41.812756 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:41.812756 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:41.812756 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:41.812756 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:41.813627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:41.813627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:41.813627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:41.813627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:41.813627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:41.813627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:41.813627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:41.813627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:41.813627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:41.813627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:41.813627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:24:41.813627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:24:41.813627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 05:24:41.813627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:41.819582 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 360/706] 10.222.0.15 () {56 vars in 1129 bytes} [Wed Aug 24 05:24:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:24:41.820725 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62c6eb-8266-4677-af19-7aae60d58715 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:41.835791 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 347/707] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:24:41.836795 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d74ce3-75f7-4ea8-8e8e-7b947c331049 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:41.955956 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d74ce3-75f7-4ea8-8e8e-7b947c331049 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:41.982648 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:41.995954 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc449d5-953f-4c9a-9c53-69c935c0cab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:41.996607 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc449d5-953f-4c9a-9c53-69c935c0cab5 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:41.996890 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc449d5-953f-4c9a-9c53-69c935c0cab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:41.997168 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc449d5-953f-4c9a-9c53-69c935c0cab5 None admin] PATH_INFO: `/v3/services` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:41.998341 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc449d5-953f-4c9a-9c53-69c935c0cab5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:41.998687 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc449d5-953f-4c9a-9c53-69c935c0cab5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:42.000321 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc449d5-953f-4c9a-9c53-69c935c0cab5 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:42.005423 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 348/708] 10.222.0.15 () {56 vars in 1124 bytes} [Wed Aug 24 05:24:41 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:42.011914 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc449d5-953f-4c9a-9c53-69c935c0cab5 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:42.073989 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62c6eb-8266-4677-af19-7aae60d58715 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:42.126079 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0faa5a7-d6be-4108-80e5-166498002114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:42.126804 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0faa5a7-d6be-4108-80e5-166498002114 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:42.127052 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0faa5a7-d6be-4108-80e5-166498002114 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:42.127351 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0faa5a7-d6be-4108-80e5-166498002114 None admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:42.128148 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0faa5a7-d6be-4108-80e5-166498002114 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:42.128508 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0faa5a7-d6be-4108-80e5-166498002114 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:42.129972 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0faa5a7-d6be-4108-80e5-166498002114 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:42.142390 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:42.165208 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 361/709] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Aug 24 05:24:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:42.167918 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0faa5a7-d6be-4108-80e5-166498002114 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:42.244713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc449d5-953f-4c9a-9c53-69c935c0cab5 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:42.292783 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb6984-d7e3-4bb5-9181-4d195cec8d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:42.293630 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb6984-d7e3-4bb5-9181-4d195cec8d27 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:42.294390 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb6984-d7e3-4bb5-9181-4d195cec8d27 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:42.294671 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb6984-d7e3-4bb5-9181-4d195cec8d27 None admin] PATH_INFO: `/v3/services` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:42.296306 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb6984-d7e3-4bb5-9181-4d195cec8d27 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:42.296665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb6984-d7e3-4bb5-9181-4d195cec8d27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:42.298736 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb6984-d7e3-4bb5-9181-4d195cec8d27 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:42.306693 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 349/710] 10.222.0.15 () {56 vars in 1124 bytes} [Wed Aug 24 05:24:42 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:24:42.307612 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb6984-d7e3-4bb5-9181-4d195cec8d27 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:42.330039 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0faa5a7-d6be-4108-80e5-166498002114 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:42.374794 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:42.430262 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb6984-d7e3-4bb5-9181-4d195cec8d27 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:42.468177 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772dcee5-7e64-4ad6-abe0-e8b23afcc6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:42.468849 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772dcee5-7e64-4ad6-abe0-e8b23afcc6d0 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:42.469242 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772dcee5-7e64-4ad6-abe0-e8b23afcc6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:42.469524 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772dcee5-7e64-4ad6-abe0-e8b23afcc6d0 None admin] PATH_INFO: `/v3/services/network` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:42.470211 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772dcee5-7e64-4ad6-abe0-e8b23afcc6d0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:42.471747 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772dcee5-7e64-4ad6-abe0-e8b23afcc6d0 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:42.473545 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2267a5c9-78c3-4f27-86a3-a83cc0495195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:42.474613 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2267a5c9-78c3-4f27-86a3-a83cc0495195 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:42.475107 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2267a5c9-78c3-4f27-86a3-a83cc0495195 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:42.475598 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2267a5c9-78c3-4f27-86a3-a83cc0495195 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:42.483952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-772dcee5-7e64-4ad6-abe0-e8b23afcc6d0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 05:24:42.483952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:42.483952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:24:42.483952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:24:42.483952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:24:42.483952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:24:42.483952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:24:42.483952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:24:42.483952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 05:24:42.483952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:42.483952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:24:42.483952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:42.483952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:42.483952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:42.483952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:42.483952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:42.483952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:42.485210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:42.485210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:42.485210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:42.485210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:42.485210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:42.485210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:42.485210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:24:42.485210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:24:42.485210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:24:42.485210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:24:42.485210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:42.485210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:42.485210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:42.485210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:42.485210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:42.485210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:42.486041 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:42.486041 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:42.486041 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:42.486041 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:42.486041 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:42.486041 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:42.486041 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:42.486041 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:42.486041 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:42.486041 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:42.486041 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:24:42.486041 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:24:42.486041 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 05:24:42.486041 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:42.486041 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 350/711] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:24:42 2022] GET /identity/v3/services/network => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:42.492347 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772dcee5-7e64-4ad6-abe0-e8b23afcc6d0 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:42.611979 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:42.617733 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2267a5c9-78c3-4f27-86a3-a83cc0495195 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:42.619390 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2267a5c9-78c3-4f27-86a3-a83cc0495195 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:42.627981 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772dcee5-7e64-4ad6-abe0-e8b23afcc6d0 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:42.648944 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:42.679698 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d0b1c-a586-47af-9eb8-7ede07bca457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:42.680315 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d0b1c-a586-47af-9eb8-7ede07bca457 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:42.681990 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d0b1c-a586-47af-9eb8-7ede07bca457 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:42.682272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d0b1c-a586-47af-9eb8-7ede07bca457 None admin] PATH_INFO: `/v3/services` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:42.682904 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38d0b1c-a586-47af-9eb8-7ede07bca457 None admin] RBAC: Adding query filter params (name=network) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:42.683207 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38d0b1c-a586-47af-9eb8-7ede07bca457 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:42.684671 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38d0b1c-a586-47af-9eb8-7ede07bca457 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:42.690623 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 351/712] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Aug 24 05:24:42 2022] GET /identity/v3/services?name=network => generated 124 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:42.699772 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d0b1c-a586-47af-9eb8-7ede07bca457 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:42.793033 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 362/713] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:24:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 624 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:24:42.801934 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2267a5c9-78c3-4f27-86a3-a83cc0495195 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:42.856978 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d0b1c-a586-47af-9eb8-7ede07bca457 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:42.920155 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deac4a25-31a3-4484-967b-7052cd9bd011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:42.920832 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac4a25-31a3-4484-967b-7052cd9bd011 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:42.921114 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac4a25-31a3-4484-967b-7052cd9bd011 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:42.921383 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac4a25-31a3-4484-967b-7052cd9bd011 None admin] PATH_INFO: `/v3/services` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:42.922023 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deac4a25-31a3-4484-967b-7052cd9bd011 None admin] RBAC: Adding query filter params (type=network) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:42.922324 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deac4a25-31a3-4484-967b-7052cd9bd011 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:42.923714 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deac4a25-31a3-4484-967b-7052cd9bd011 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:42.932826 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 352/714] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Aug 24 05:24:42 2022] GET /identity/v3/services?type=network => generated 124 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:24:43.020837 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:43.025311 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2267a5c9-78c3-4f27-86a3-a83cc0495195 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:43.079078 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868fc7ae-0643-45a6-a39d-45a4fdcbd2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:43.080093 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868fc7ae-0643-45a6-a39d-45a4fdcbd2ec None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:43.080602 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868fc7ae-0643-45a6-a39d-45a4fdcbd2ec None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:43.081071 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868fc7ae-0643-45a6-a39d-45a4fdcbd2ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:43.098603 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868fc7ae-0643-45a6-a39d-45a4fdcbd2ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:43.100048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868fc7ae-0643-45a6-a39d-45a4fdcbd2ec None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:43.118719 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-868fc7ae-0643-45a6-a39d-45a4fdcbd2ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:43.118719 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:43.118719 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:43.118719 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:43.118719 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:43.118719 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:43.118719 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:43.118719 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:43.118719 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:43.118719 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:43.118719 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:43.118719 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:43.118719 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:43.118719 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:43.118719 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:43.118719 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:43.119764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:43.119764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:43.119764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:43.119764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:43.119764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:43.119764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:43.119764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:43.119764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:43.119764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:43.119764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:43.119764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:43.119764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:43.119764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:43.119764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:43.119764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:43.119764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:43.119764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:43.120697 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:43.120697 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:43.120697 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:43.120697 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 363/715] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:43.125755 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deac4a25-31a3-4484-967b-7052cd9bd011 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:43.174672 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:43.236530 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deac4a25-31a3-4484-967b-7052cd9bd011 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:43.273495 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b03de0-0c10-4496-bac7-2d19ed470789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:43.274124 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b03de0-0c10-4496-bac7-2d19ed470789 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:43.274405 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b03de0-0c10-4496-bac7-2d19ed470789 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:43.274679 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b03de0-0c10-4496-bac7-2d19ed470789 None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:43.275329 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b03de0-0c10-4496-bac7-2d19ed470789 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:43.275632 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b03de0-0c10-4496-bac7-2d19ed470789 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:43.277850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224f00b-43d4-4c2e-b443-73279ff75695 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:43.278309 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224f00b-43d4-4c2e-b443-73279ff75695 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:43.278386 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224f00b-43d4-4c2e-b443-73279ff75695 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:43.279978 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b03de0-0c10-4496-bac7-2d19ed470789 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:43.280097 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 364/716] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:43.287209 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9e393-2dbd-4ed6-bce5-21dd19824bcc None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:43.287486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9e393-2dbd-4ed6-bce5-21dd19824bcc None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:43.287767 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9e393-2dbd-4ed6-bce5-21dd19824bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:43.302028 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 353/717] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:43.308318 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b03de0-0c10-4496-bac7-2d19ed470789 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:43.312841 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-26b9e393-2dbd-4ed6-bce5-21dd19824bcc None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:43.314100 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-26b9e393-2dbd-4ed6-bce5-21dd19824bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:43.446411 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 365/718] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:24:43.481258 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b03de0-0c10-4496-bac7-2d19ed470789 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:43.488958 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:43.517158 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d92ac-719b-4a79-ad62-a5024d7a4c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:43.517795 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d92ac-719b-4a79-ad62-a5024d7a4c4f None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:43.518078 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d92ac-719b-4a79-ad62-a5024d7a4c4f None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:43.518353 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d92ac-719b-4a79-ad62-a5024d7a4c4f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:43.525827 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656d92ac-719b-4a79-ad62-a5024d7a4c4f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:43.527416 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656d92ac-719b-4a79-ad62-a5024d7a4c4f None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:43.540442 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-656d92ac-719b-4a79-ad62-a5024d7a4c4f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:43.540442 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:43.540442 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:43.540442 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:43.540442 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:43.540442 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:43.540442 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:43.540442 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:43.540442 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:43.540442 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:43.540442 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:43.540442 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:43.540442 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:24:43.540442 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:24:43.540442 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:24:43.541557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:24:43.541557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:43.541557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:43.541557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:43.541557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:43.541557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:43.541557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:43.541557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:43.541557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:43.541557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:43.541557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:43.541557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:43.541557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:43.541557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:43.541557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:43.541557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:43.541557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:43.543560 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:24:43.543560 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:24:43.543560 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:24:43.543560 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:24:43.543560 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:24:43.543560 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:24:43.543560 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:43.543560 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:43.549826 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 354/719] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:24:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:43.556506 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9e393-2dbd-4ed6-bce5-21dd19824bcc None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:43.692530 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:43.715533 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9e393-2dbd-4ed6-bce5-21dd19824bcc None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:43.756755 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b9c1f-dece-4ae2-acf0-4629bdb4d903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:43.758920 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b9c1f-dece-4ae2-acf0-4629bdb4d903 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:43.758920 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b9c1f-dece-4ae2-acf0-4629bdb4d903 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:43.758920 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b9c1f-dece-4ae2-acf0-4629bdb4d903 None admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:43.758920 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313b9c1f-dece-4ae2-acf0-4629bdb4d903 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:43.759192 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313b9c1f-dece-4ae2-acf0-4629bdb4d903 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:43.760487 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313b9c1f-dece-4ae2-acf0-4629bdb4d903 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:43.775326 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 366/720] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Aug 24 05:24:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:24:43.782103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d92ac-719b-4a79-ad62-a5024d7a4c4f None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:43.884676 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b9c1f-dece-4ae2-acf0-4629bdb4d903 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:43.950149 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d92ac-719b-4a79-ad62-a5024d7a4c4f None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:44.030321 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633ebb9d-4e9e-41be-9f8c-8e2c5ea360e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:44.032510 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ebb9d-4e9e-41be-9f8c-8e2c5ea360e9 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:44.033087 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ebb9d-4e9e-41be-9f8c-8e2c5ea360e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:44.033561 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ebb9d-4e9e-41be-9f8c-8e2c5ea360e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:44.034606 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633ebb9d-4e9e-41be-9f8c-8e2c5ea360e9 None admin] RBAC: Adding query filter params (role.id=e013ad4d436f4cb0a9bf7debb7d13dc7, scope.project.id=59592706ae9442e08353158c625e9581, user.id=fbecffb800584c5786813accd061421b) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:44.035147 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633ebb9d-4e9e-41be-9f8c-8e2c5ea360e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:44.036847 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633ebb9d-4e9e-41be-9f8c-8e2c5ea360e9 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:44.043408 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 355/721] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:24:43 2022] GET /identity/v3/role_assignments?user.id=fbecffb800584c5786813accd061421b&scope.project.id=59592706ae9442e08353158c625e9581&role.id=e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 259 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:44.073849 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b9c1f-dece-4ae2-acf0-4629bdb4d903 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:44.152847 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc67983c-0d81-4908-8f28-6c8099051871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:44.153815 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67983c-0d81-4908-8f28-6c8099051871 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:44.154241 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67983c-0d81-4908-8f28-6c8099051871 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:44.155202 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67983c-0d81-4908-8f28-6c8099051871 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:44.182936 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5a686-448f-4fdf-9e00-aba4390878a6 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:44.183583 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5a686-448f-4fdf-9e00-aba4390878a6 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:44.184084 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5a686-448f-4fdf-9e00-aba4390878a6 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:44.186348 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 356/722] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:24:44.189458 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:44.191983 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4685d2e-fffe-4e17-ab5d-3749f8f53624 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:44.192272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4685d2e-fffe-4e17-ab5d-3749f8f53624 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:44.192593 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4685d2e-fffe-4e17-ab5d-3749f8f53624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:44.197212 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc67983c-0d81-4908-8f28-6c8099051871 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:24:44.204293 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-fc67983c-0d81-4908-8f28-6c8099051871 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:24:44.204293 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:44.204293 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:44.204293 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:44.204293 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:44.204293 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:44.204293 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:44.204293 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:44.204293 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:44.204293 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:44.204293 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:44.204293 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:44.204293 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:24:44.204293 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:24:44.204293 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:24:44.204293 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:24:44.205278 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:24:44.205278 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:24:44.205278 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:24:44.205278 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:24:44.205278 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:44.205278 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:44.205278 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:44.205278 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:44.205278 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:44.205278 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:44.205278 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:44.205278 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:44.205278 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:44.205278 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:44.205278 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:44.205278 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:44.206103 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:44.206103 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:44.206103 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:44.206103 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:44.206103 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:24:44.206103 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:24:44.206103 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:24:44.206103 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:24:44.206103 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:24:44.206103 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:24:44.206103 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:24:44.206103 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:44.206737 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 367/723] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Aug 24 05:24:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:24:44.218426 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc67983c-0d81-4908-8f28-6c8099051871 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:44.227575 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-d4685d2e-fffe-4e17-ab5d-3749f8f53624 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:44.228858 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-d4685d2e-fffe-4e17-ab5d-3749f8f53624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:44.266276 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:44.356503 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 357/724] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:24:44.396289 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc67983c-0d81-4908-8f28-6c8099051871 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:44.448611 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9994f6-b984-4304-928a-f52f015a9abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:44.449232 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9994f6-b984-4304-928a-f52f015a9abb None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:44.449510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9994f6-b984-4304-928a-f52f015a9abb None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:44.449782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9994f6-b984-4304-928a-f52f015a9abb None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:44.450452 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9994f6-b984-4304-928a-f52f015a9abb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:44.450761 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9994f6-b984-4304-928a-f52f015a9abb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:44.452806 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9994f6-b984-4304-928a-f52f015a9abb None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:44.465026 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 368/725] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:24:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:44.470795 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4685d2e-fffe-4e17-ab5d-3749f8f53624 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:44.615240 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4685d2e-fffe-4e17-ab5d-3749f8f53624 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:44.654641 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5341143-e021-4b42-8a27-688ba977225c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:44.655685 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5341143-e021-4b42-8a27-688ba977225c None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:44.656452 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5341143-e021-4b42-8a27-688ba977225c None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:44.656943 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5341143-e021-4b42-8a27-688ba977225c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:44.668011 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5341143-e021-4b42-8a27-688ba977225c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:44.669771 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5341143-e021-4b42-8a27-688ba977225c None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:44.686543 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-c5341143-e021-4b42-8a27-688ba977225c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:44.686543 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:44.686543 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:44.686543 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:44.686543 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:44.686543 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:44.686543 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:44.686543 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:44.686543 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:44.686543 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:44.686543 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:44.686543 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:44.686543 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:44.686543 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:44.686543 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:44.686543 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:44.687697 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:44.687697 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:44.687697 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:44.687697 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:44.687697 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:44.687697 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:44.687697 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:44.687697 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:44.687697 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:44.687697 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:44.687697 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:44.687697 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:44.687697 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:44.687697 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:44.687697 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:44.687697 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:44.687697 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:44.693158 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:44.693158 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:44.693158 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:44.693382 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9994f6-b984-4304-928a-f52f015a9abb None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:44.693457 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 358/726] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:44.697062 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5341143-e021-4b42-8a27-688ba977225c None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:44.718861 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:44.765512 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:44.816688 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9994f6-b984-4304-928a-f52f015a9abb None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:44.893222 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5341143-e021-4b42-8a27-688ba977225c None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:44.923356 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460bfe80-962e-42a7-8a8f-69d0e91abec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:44.924452 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460bfe80-962e-42a7-8a8f-69d0e91abec9 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:44.924814 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460bfe80-962e-42a7-8a8f-69d0e91abec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:44.925238 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460bfe80-962e-42a7-8a8f-69d0e91abec9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:44.954275 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460bfe80-962e-42a7-8a8f-69d0e91abec9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:24:44.959546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-460bfe80-962e-42a7-8a8f-69d0e91abec9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:24:44.959546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:44.959546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:44.959546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:44.959546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:44.959546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:44.959546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:44.959546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:44.959546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:44.959546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:44.959546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:44.959546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:44.959546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:24:44.959546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:24:44.959546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:24:44.959546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:24:44.960489 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:24:44.960489 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:24:44.960489 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:24:44.960489 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:24:44.960489 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:44.960489 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:44.960489 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:44.960489 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:44.960489 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:44.960489 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:44.960489 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:44.960489 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:44.960489 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:44.960489 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:44.960489 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:44.960489 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:44.961331 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:44.961331 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:44.961331 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:44.961331 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:44.961331 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:24:44.961331 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:24:44.961331 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:24:44.961331 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:24:44.961331 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:24:44.961331 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:24:44.961331 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:24:44.961331 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:44.961962 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 369/727] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Aug 24 05:24:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:44.966666 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460bfe80-962e-42a7-8a8f-69d0e91abec9 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:44.983417 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b82a49-0b6e-48f8-8d85-d4560d34e00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:44.984192 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b82a49-0b6e-48f8-8d85-d4560d34e00b None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:44.985628 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b82a49-0b6e-48f8-8d85-d4560d34e00b None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:44.985960 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b82a49-0b6e-48f8-8d85-d4560d34e00b None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:44.986625 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b82a49-0b6e-48f8-8d85-d4560d34e00b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:44.986934 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b82a49-0b6e-48f8-8d85-d4560d34e00b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:44.988352 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b82a49-0b6e-48f8-8d85-d4560d34e00b None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:45.012693 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 359/728] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:24:45.018513 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b82a49-0b6e-48f8-8d85-d4560d34e00b None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:45.127102 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:45.135438 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460bfe80-962e-42a7-8a8f-69d0e91abec9 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:45.159397 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:45.167786 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b82a49-0b6e-48f8-8d85-d4560d34e00b None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:45.172066 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55e158-6f6d-4700-a05b-8172cc53fcb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:45.172374 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55e158-6f6d-4700-a05b-8172cc53fcb3 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:45.172709 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55e158-6f6d-4700-a05b-8172cc53fcb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:45.173640 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55e158-6f6d-4700-a05b-8172cc53fcb3 None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:45.173730 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f55e158-6f6d-4700-a05b-8172cc53fcb3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:45.174107 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f55e158-6f6d-4700-a05b-8172cc53fcb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:45.175502 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f55e158-6f6d-4700-a05b-8172cc53fcb3 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:45.196133 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 370/729] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Aug 24 05:24:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:45.197753 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039e83d0-9d16-4394-9e0f-364d05553c36 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:45.198224 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039e83d0-9d16-4394-9e0f-364d05553c36 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:45.198755 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039e83d0-9d16-4394-9e0f-364d05553c36 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:45.200582 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 371/730] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:45.201573 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039e83d0-9d16-4394-9e0f-364d05553c36 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:45.212123 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64888b7c-04f4-4376-8c49-9a6fc93a4f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:45.212790 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64888b7c-04f4-4376-8c49-9a6fc93a4f24 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:45.213071 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64888b7c-04f4-4376-8c49-9a6fc93a4f24 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:45.213348 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64888b7c-04f4-4376-8c49-9a6fc93a4f24 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:45.222996 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64888b7c-04f4-4376-8c49-9a6fc93a4f24 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:45.224777 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64888b7c-04f4-4376-8c49-9a6fc93a4f24 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:45.237677 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-64888b7c-04f4-4376-8c49-9a6fc93a4f24 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 05:24:45.237677 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:45.237677 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:45.237677 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:45.237677 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:45.237677 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:45.237677 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:45.237677 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:45.237677 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:45.237677 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:45.237677 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:45.237677 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:45.237677 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:24:45.237677 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:24:45.237677 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:24:45.237677 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:24:45.239333 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:45.239333 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:45.239333 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:24:45.239333 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:45.239333 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:24:45.239333 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:45.239333 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:45.239333 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:45.239333 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:45.239333 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:45.239333 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:45.239333 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:45.239333 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:45.239333 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:45.239333 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:45.239333 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:45.239333 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:45.239333 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:45.240391 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:45.240391 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:45.240391 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:24:45.240391 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:24:45.240391 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:24:45.240391 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:24:45.240391 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:24:45.240391 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:24:45.240391 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 05:24:45.240391 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:45.243999 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 360/731] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Aug 24 05:24:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:24:45.244919 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eedf85-57cb-4758-a8dd-5526a1c01ea0 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:45.245204 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eedf85-57cb-4758-a8dd-5526a1c01ea0 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:45.245481 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eedf85-57cb-4758-a8dd-5526a1c01ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:45.267595 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-13eedf85-57cb-4758-a8dd-5526a1c01ea0 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:45.268930 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-13eedf85-57cb-4758-a8dd-5526a1c01ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:45.344922 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039e83d0-9d16-4394-9e0f-364d05553c36 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:45.356484 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 361/732] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:24:45.357573 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eedf85-57cb-4758-a8dd-5526a1c01ea0 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:45.408966 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1c4d2c-b554-4a59-bfad-71062dee7655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:45.410385 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c4d2c-b554-4a59-bfad-71062dee7655 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:45.411700 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c4d2c-b554-4a59-bfad-71062dee7655 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:45.412348 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c4d2c-b554-4a59-bfad-71062dee7655 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:45.437502 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1c4d2c-b554-4a59-bfad-71062dee7655 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:45.440598 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1c4d2c-b554-4a59-bfad-71062dee7655 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:45.469716 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-be1c4d2c-b554-4a59-bfad-71062dee7655 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:45.469716 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:45.469716 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:45.469716 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:45.469716 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:45.469716 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:45.469716 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:45.469716 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:45.469716 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:45.469716 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:45.469716 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:45.469716 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:45.469716 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:45.469716 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:45.469716 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:45.469716 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:45.470943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:45.470943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:45.470943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:45.470943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:45.470943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:45.470943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:45.470943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:45.470943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:45.470943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:45.470943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:45.470943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:45.470943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:45.470943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:45.470943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:45.470943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:45.470943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:45.470943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:45.472070 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:45.472070 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:45.472070 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:45.474485 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 372/733] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:45.479330 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:45.480682 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1c4d2c-b554-4a59-bfad-71062dee7655 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:45.543242 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:45.550531 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eedf85-57cb-4758-a8dd-5526a1c01ea0 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:45.594180 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2ee4d-5d2c-4d2d-8b6e-bfcae788a09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:45.594895 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2ee4d-5d2c-4d2d-8b6e-bfcae788a09f None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:45.595255 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2ee4d-5d2c-4d2d-8b6e-bfcae788a09f None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:45.595600 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2ee4d-5d2c-4d2d-8b6e-bfcae788a09f None admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:45.596435 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e2ee4d-5d2c-4d2d-8b6e-bfcae788a09f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:45.596822 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e2ee4d-5d2c-4d2d-8b6e-bfcae788a09f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:45.598331 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e2ee4d-5d2c-4d2d-8b6e-bfcae788a09f None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:45.638268 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1c4d2c-b554-4a59-bfad-71062dee7655 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:45.678942 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a858e8-0e26-4290-9ffa-eca64c30c411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:45.679467 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a858e8-0e26-4290-9ffa-eca64c30c411 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:45.679750 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a858e8-0e26-4290-9ffa-eca64c30c411 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:45.680027 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a858e8-0e26-4290-9ffa-eca64c30c411 None admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:45.680742 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a858e8-0e26-4290-9ffa-eca64c30c411 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:45.681051 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a858e8-0e26-4290-9ffa-eca64c30c411 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:45.682480 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a858e8-0e26-4290-9ffa-eca64c30c411 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:45.696429 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 373/734] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:45.697556 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc582be2-5218-4c8d-bd15-2c05d815b246 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:45.697842 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc582be2-5218-4c8d-bd15-2c05d815b246 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:45.698125 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc582be2-5218-4c8d-bd15-2c05d815b246 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:45.699648 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 374/735] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:45.700412 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc582be2-5218-4c8d-bd15-2c05d815b246 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:45.742199 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 362/736] 10.222.0.15 () {56 vars in 1135 bytes} [Wed Aug 24 05:24:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:45.743047 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba10ca1-15a3-47a3-83c0-fd073083e8af None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:45.743331 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba10ca1-15a3-47a3-83c0-fd073083e8af None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:45.743607 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba10ca1-15a3-47a3-83c0-fd073083e8af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:45.763431 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-aba10ca1-15a3-47a3-83c0-fd073083e8af None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:45.765312 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-aba10ca1-15a3-47a3-83c0-fd073083e8af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:45.813711 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:45.821012 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc582be2-5218-4c8d-bd15-2c05d815b246 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:45.848694 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 363/737] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:24:45.849522 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba10ca1-15a3-47a3-83c0-fd073083e8af None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:45.861978 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a7c92f-4c4f-4f38-8632-bae57fe6c935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:45.862787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a7c92f-4c4f-4f38-8632-bae57fe6c935 None admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:45.863167 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a7c92f-4c4f-4f38-8632-bae57fe6c935 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:45.863590 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a7c92f-4c4f-4f38-8632-bae57fe6c935 None admin] PATH_INFO: `/v3/services` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:45.864962 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a7c92f-4c4f-4f38-8632-bae57fe6c935 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:45.866757 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a7c92f-4c4f-4f38-8632-bae57fe6c935 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:45.903661 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:45.956446 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba10ca1-15a3-47a3-83c0-fd073083e8af None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:45.970355 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 375/738] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Aug 24 05:24:45 2022] POST /identity/v3/services => generated 251 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:24:45.973040 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a7c92f-4c4f-4f38-8632-bae57fe6c935 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:46.038605 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585e9139-dda0-40f9-bc68-bb505b3705b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:46.041392 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585e9139-dda0-40f9-bc68-bb505b3705b5 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:46.041869 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585e9139-dda0-40f9-bc68-bb505b3705b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:46.043146 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585e9139-dda0-40f9-bc68-bb505b3705b5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:46.058338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585e9139-dda0-40f9-bc68-bb505b3705b5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:46.061195 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585e9139-dda0-40f9-bc68-bb505b3705b5 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:46.084725 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-585e9139-dda0-40f9-bc68-bb505b3705b5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 05:24:46.084725 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:46.084725 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:46.084725 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:46.084725 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:46.084725 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:46.084725 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:46.084725 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:46.084725 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:46.084725 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:46.084725 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:46.084725 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:46.084725 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:24:46.084725 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:24:46.084725 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:24:46.084725 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:24:46.085728 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:46.085728 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:46.085728 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:24:46.085728 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:46.085728 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:24:46.085728 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:46.085728 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:46.085728 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:46.085728 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:46.085728 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:46.085728 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:46.085728 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:46.085728 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:46.085728 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:46.085728 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:46.085728 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:46.085728 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:46.085728 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:46.086675 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:46.086675 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:46.086675 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:24:46.086675 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:24:46.086675 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:24:46.086675 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:24:46.086675 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:24:46.086675 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:24:46.086675 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 05:24:46.086675 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:46.090577 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 364/739] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Aug 24 05:24:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:46.097030 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585e9139-dda0-40f9-bc68-bb505b3705b5 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:46.164898 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a7c92f-4c4f-4f38-8632-bae57fe6c935 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:46.220359 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a62ea9-1d46-4020-a88f-bfff88faa72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:46.221103 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a62ea9-1d46-4020-a88f-bfff88faa72e None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:46.221392 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a62ea9-1d46-4020-a88f-bfff88faa72e None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:46.221583 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a62ea9-1d46-4020-a88f-bfff88faa72e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:46.227699 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585e9139-dda0-40f9-bc68-bb505b3705b5 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:46.229861 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:46.250623 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:46.279828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e768f1d0-deef-4dde-8f64-3dd6813f42f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:46.280662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768f1d0-deef-4dde-8f64-3dd6813f42f9 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:46.281201 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768f1d0-deef-4dde-8f64-3dd6813f42f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:46.281487 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768f1d0-deef-4dde-8f64-3dd6813f42f9 None admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:46.282692 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e768f1d0-deef-4dde-8f64-3dd6813f42f9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:46.283157 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e768f1d0-deef-4dde-8f64-3dd6813f42f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:46.285025 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e768f1d0-deef-4dde-8f64-3dd6813f42f9 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:46.299639 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a62ea9-1d46-4020-a88f-bfff88faa72e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:46.301296 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a62ea9-1d46-4020-a88f-bfff88faa72e None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:46.308202 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 365/740] 10.222.0.15 () {56 vars in 1149 bytes} [Wed Aug 24 05:24:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:46.313140 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e768f1d0-deef-4dde-8f64-3dd6813f42f9 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:46.406520 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 376/741] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:24:45 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:24:46.408180 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da0c462-6cfa-4010-a844-be3190017dee None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:46.408547 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da0c462-6cfa-4010-a844-be3190017dee None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:46.408898 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da0c462-6cfa-4010-a844-be3190017dee None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:46.410392 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 377/742] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:46.414953 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da0c462-6cfa-4010-a844-be3190017dee None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:46.430036 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e768f1d0-deef-4dde-8f64-3dd6813f42f9 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:46.468608 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f93976b-3ec8-42c3-9bb6-72760ea6c955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:46.469570 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93976b-3ec8-42c3-9bb6-72760ea6c955 None admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:46.470163 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93976b-3ec8-42c3-9bb6-72760ea6c955 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:46.470670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93976b-3ec8-42c3-9bb6-72760ea6c955 None admin] PATH_INFO: `/v3/services` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:46.471469 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f93976b-3ec8-42c3-9bb6-72760ea6c955 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:46.473253 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f93976b-3ec8-42c3-9bb6-72760ea6c955 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:46.473694 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:46.517726 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da0c462-6cfa-4010-a844-be3190017dee None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:46.551536 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b65a3-3e97-4c20-a9d6-0debd3a60e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:46.552143 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b65a3-3e97-4c20-a9d6-0debd3a60e22 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:46.552528 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b65a3-3e97-4c20-a9d6-0debd3a60e22 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:46.552728 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b65a3-3e97-4c20-a9d6-0debd3a60e22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:46.580527 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 366/743] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Aug 24 05:24:46 2022] POST /identity/v3/services => generated 240 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:24:46.580527 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f93976b-3ec8-42c3-9bb6-72760ea6c955 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:46.605940 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:46.640130 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b65a3-3e97-4c20-a9d6-0debd3a60e22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:46.641993 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b65a3-3e97-4c20-a9d6-0debd3a60e22 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:46.708740 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:46.710944 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f93976b-3ec8-42c3-9bb6-72760ea6c955 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:46.751499 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05e3c0a-db1f-401b-954c-6967b9c7fcdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:46.752121 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e3c0a-db1f-401b-954c-6967b9c7fcdf None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:46.752502 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e3c0a-db1f-401b-954c-6967b9c7fcdf None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:46.752703 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e3c0a-db1f-401b-954c-6967b9c7fcdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:46.763878 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05e3c0a-db1f-401b-954c-6967b9c7fcdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:46.765498 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05e3c0a-db1f-401b-954c-6967b9c7fcdf None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:46.766889 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 378/744] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:24:46 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:24:46.769070 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd414b6b-fd46-46dd-b0ab-33f6274c5667 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:46.769358 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd414b6b-fd46-46dd-b0ab-33f6274c5667 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:46.769637 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd414b6b-fd46-46dd-b0ab-33f6274c5667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:46.782917 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-d05e3c0a-db1f-401b-954c-6967b9c7fcdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:46.782917 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:46.782917 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:46.782917 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:46.782917 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:46.782917 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:46.782917 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:46.782917 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:46.782917 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:46.782917 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:46.782917 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:46.782917 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:46.782917 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:46.782917 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:46.782917 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:46.782917 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:46.788493 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:46.788493 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:46.788493 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:46.788493 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:46.788493 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:46.788493 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:46.788493 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:46.788493 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:46.788493 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:46.788493 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:46.788493 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:46.788493 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:46.788493 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:46.788493 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:46.788493 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:46.788493 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:46.788493 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:46.789409 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:46.789409 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:46.789409 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:46.789409 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 367/745] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:46.789409 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05e3c0a-db1f-401b-954c-6967b9c7fcdf None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:46.829151 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-cd414b6b-fd46-46dd-b0ab-33f6274c5667 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:46.831272 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-cd414b6b-fd46-46dd-b0ab-33f6274c5667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:46.888550 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:46.916018 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 379/746] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:24:46.926314 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd414b6b-fd46-46dd-b0ab-33f6274c5667 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:46.928320 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05e3c0a-db1f-401b-954c-6967b9c7fcdf None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:46.971264 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91aa1f-e1aa-46b5-bb70-75ddb3fda612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:46.971983 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91aa1f-e1aa-46b5-bb70-75ddb3fda612 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:46.972556 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91aa1f-e1aa-46b5-bb70-75ddb3fda612 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:46.972905 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91aa1f-e1aa-46b5-bb70-75ddb3fda612 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:46.986384 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df91aa1f-e1aa-46b5-bb70-75ddb3fda612 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:46.988123 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df91aa1f-e1aa-46b5-bb70-75ddb3fda612 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:47.004821 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-df91aa1f-e1aa-46b5-bb70-75ddb3fda612 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:47.004821 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:47.004821 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:47.004821 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:47.004821 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:47.004821 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:47.004821 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:47.004821 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:47.004821 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:47.004821 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:47.004821 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:47.004821 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:47.004821 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:47.004821 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:47.004821 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:47.004821 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:47.005974 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:47.005974 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:47.005974 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:47.005974 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:47.005974 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:47.005974 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:47.005974 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:47.005974 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:47.005974 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:47.005974 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:47.005974 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:47.005974 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:47.005974 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:47.005974 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:47.005974 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:47.005974 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:47.005974 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:47.007064 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:47.007064 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:47.007064 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:47.009092 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 368/747] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:47.021467 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91aa1f-e1aa-46b5-bb70-75ddb3fda612 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:47.057452 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:47.073559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd414b6b-fd46-46dd-b0ab-33f6274c5667 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:47.111918 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b6db0b-2e22-4c2f-986b-22c446824255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:47.112881 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b6db0b-2e22-4c2f-986b-22c446824255 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:47.113343 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b6db0b-2e22-4c2f-986b-22c446824255 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:47.113904 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b6db0b-2e22-4c2f-986b-22c446824255 None admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:47.114733 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b6db0b-2e22-4c2f-986b-22c446824255 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:47.115217 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b6db0b-2e22-4c2f-986b-22c446824255 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:47.116789 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b6db0b-2e22-4c2f-986b-22c446824255 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:47.129841 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 380/748] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:47.135559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b6db0b-2e22-4c2f-986b-22c446824255 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:47.251650 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b6db0b-2e22-4c2f-986b-22c446824255 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:47.255154 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91aa1f-e1aa-46b5-bb70-75ddb3fda612 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:47.289632 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7569b429-362a-4d3d-9269-4410b768bdcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:47.290614 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7569b429-362a-4d3d-9269-4410b768bdcf None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:47.291084 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7569b429-362a-4d3d-9269-4410b768bdcf None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:47.291525 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7569b429-362a-4d3d-9269-4410b768bdcf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:47.302963 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7569b429-362a-4d3d-9269-4410b768bdcf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:47.304873 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7569b429-362a-4d3d-9269-4410b768bdcf None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:47.311448 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:47.327091 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-7569b429-362a-4d3d-9269-4410b768bdcf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:47.327091 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:47.327091 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:47.327091 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:47.327091 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:47.327091 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:47.327091 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:47.327091 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:47.327091 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:47.327091 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:47.327091 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:47.327091 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:47.327091 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:24:47.327091 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:24:47.327091 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:24:47.328129 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:24:47.328129 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:47.328129 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:47.328129 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:47.328129 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:47.328129 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:47.328129 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:47.328129 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:47.328129 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:47.328129 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:47.328129 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:47.328129 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:47.328129 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:47.328129 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:47.328129 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:47.328129 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:47.328129 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:47.329061 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:24:47.329061 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:24:47.329061 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:24:47.329061 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:24:47.329061 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:24:47.329061 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:24:47.329061 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:47.329061 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:47.337673 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 381/749] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:24:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:24:47.340846 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ec2c0-51b6-4ca7-a2fd-30b854f68ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:47.341629 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ec2c0-51b6-4ca7-a2fd-30b854f68ef2 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:47.342654 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ec2c0-51b6-4ca7-a2fd-30b854f68ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:47.343457 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ec2c0-51b6-4ca7-a2fd-30b854f68ef2 None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:47.344342 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5ec2c0-51b6-4ca7-a2fd-30b854f68ef2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:47.345353 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7569b429-362a-4d3d-9269-4410b768bdcf None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:47.346189 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5ec2c0-51b6-4ca7-a2fd-30b854f68ef2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:47.350109 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5ec2c0-51b6-4ca7-a2fd-30b854f68ef2 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:47.359397 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:47.376561 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 369/750] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:47.382226 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ec2c0-51b6-4ca7-a2fd-30b854f68ef2 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:47.481064 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7569b429-362a-4d3d-9269-4410b768bdcf None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:47.509915 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ec2c0-51b6-4ca7-a2fd-30b854f68ef2 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:47.519641 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252f47b-bf10-4504-a048-dc11f9de1062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:47.520585 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252f47b-bf10-4504-a048-dc11f9de1062 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:47.521016 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252f47b-bf10-4504-a048-dc11f9de1062 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:47.521439 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252f47b-bf10-4504-a048-dc11f9de1062 None admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:47.522348 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252f47b-bf10-4504-a048-dc11f9de1062 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:47.522859 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252f47b-bf10-4504-a048-dc11f9de1062 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:47.524542 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252f47b-bf10-4504-a048-dc11f9de1062 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:47.538239 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 382/751] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Aug 24 05:24:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:47.539353 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252f47b-bf10-4504-a048-dc11f9de1062 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:47.557211 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c7c38-0a6b-431c-b6f4-7ed451fd454c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:47.558013 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c7c38-0a6b-431c-b6f4-7ed451fd454c None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:47.558300 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c7c38-0a6b-431c-b6f4-7ed451fd454c None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:47.558755 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c7c38-0a6b-431c-b6f4-7ed451fd454c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:47.567027 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c7c38-0a6b-431c-b6f4-7ed451fd454c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:47.568620 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c7c38-0a6b-431c-b6f4-7ed451fd454c None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:47.583208 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-e25c7c38-0a6b-431c-b6f4-7ed451fd454c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:47.583208 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:47.583208 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:47.583208 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:47.583208 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:47.583208 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:47.583208 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:47.583208 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:47.583208 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:47.583208 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:47.583208 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:47.583208 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:47.583208 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:24:47.583208 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:24:47.583208 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:24:47.584173 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:24:47.584173 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:47.584173 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:47.584173 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:47.584173 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:47.584173 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:47.584173 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:47.584173 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:47.584173 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:47.584173 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:47.584173 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:47.584173 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:47.584173 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:47.584173 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:47.584173 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:47.584173 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:47.584173 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:47.585100 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:24:47.585100 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:24:47.585100 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:24:47.585100 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:24:47.585100 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:24:47.585100 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:24:47.585100 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:47.585100 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:47.590035 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 370/752] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:24:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:47.591405 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c7c38-0a6b-431c-b6f4-7ed451fd454c None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:47.618713 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:47.640084 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:47.656793 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252f47b-bf10-4504-a048-dc11f9de1062 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:47.693365 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff47f25-7fa4-42fa-82d4-4a743a562c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:47.694404 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff47f25-7fa4-42fa-82d4-4a743a562c63 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:47.694961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff47f25-7fa4-42fa-82d4-4a743a562c63 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:47.695490 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff47f25-7fa4-42fa-82d4-4a743a562c63 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:47.707616 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff47f25-7fa4-42fa-82d4-4a743a562c63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:24:47.708531 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c7c38-0a6b-431c-b6f4-7ed451fd454c None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:47.714376 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-cff47f25-7fa4-42fa-82d4-4a743a562c63 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:24:47.714376 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:47.714376 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:47.714376 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:47.714376 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:47.714376 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:47.714376 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:47.714376 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:47.714376 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:47.714376 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:47.714376 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:47.714376 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:47.714376 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:24:47.714376 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:24:47.714376 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:24:47.714376 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:24:47.715287 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:24:47.715287 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:24:47.715287 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:24:47.715287 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:24:47.715287 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:47.715287 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:47.715287 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:47.715287 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:47.715287 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:47.715287 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:47.715287 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:47.715287 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:47.715287 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:47.715287 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:47.715287 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:47.715287 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:47.716134 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:47.716134 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:47.716134 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:47.716134 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:47.716134 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:24:47.716134 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:24:47.716134 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:24:47.716134 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:24:47.716134 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:24:47.716134 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:24:47.716134 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:24:47.716134 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:47.718920 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 383/753] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Aug 24 05:24:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:47.720067 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff47f25-7fa4-42fa-82d4-4a743a562c63 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:47.747647 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789c8b8-830a-42f5-bbb1-2a34702f28f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:47.748348 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789c8b8-830a-42f5-bbb1-2a34702f28f9 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:47.748686 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789c8b8-830a-42f5-bbb1-2a34702f28f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:47.748969 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789c8b8-830a-42f5-bbb1-2a34702f28f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:47.749714 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9789c8b8-830a-42f5-bbb1-2a34702f28f9 None admin] RBAC: Adding query filter params (role.id=e013ad4d436f4cb0a9bf7debb7d13dc7, scope.project.id=59592706ae9442e08353158c625e9581, user.id=bb2b6cae4a004539a95fc7dd558db255) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:47.753662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9789c8b8-830a-42f5-bbb1-2a34702f28f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:47.753662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9789c8b8-830a-42f5-bbb1-2a34702f28f9 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:47.764937 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 371/754] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:24:47 2022] GET /identity/v3/role_assignments?user.id=bb2b6cae4a004539a95fc7dd558db255&scope.project.id=59592706ae9442e08353158c625e9581&role.id=e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:47.766135 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789c8b8-830a-42f5-bbb1-2a34702f28f9 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:47.898425 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff47f25-7fa4-42fa-82d4-4a743a562c63 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:47.960954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:47.965076 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789c8b8-830a-42f5-bbb1-2a34702f28f9 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:47.969310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:47.973100 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cdd098-f0f3-46bc-8aa9-41e9f311c996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:47.973717 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cdd098-f0f3-46bc-8aa9-41e9f311c996 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:47.974001 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cdd098-f0f3-46bc-8aa9-41e9f311c996 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:47.974278 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cdd098-f0f3-46bc-8aa9-41e9f311c996 None admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:47.974981 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cdd098-f0f3-46bc-8aa9-41e9f311c996 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:47.975291 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cdd098-f0f3-46bc-8aa9-41e9f311c996 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:47.984561 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cdd098-f0f3-46bc-8aa9-41e9f311c996 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:48.007082 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 384/755] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Aug 24 05:24:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:48.007748 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8c47a-1561-4286-b4bb-cb5a884ae318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:48.008639 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8c47a-1561-4286-b4bb-cb5a884ae318 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:48.009122 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8c47a-1561-4286-b4bb-cb5a884ae318 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:48.009645 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8c47a-1561-4286-b4bb-cb5a884ae318 None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:48.010543 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa8c47a-1561-4286-b4bb-cb5a884ae318 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:48.011270 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa8c47a-1561-4286-b4bb-cb5a884ae318 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:48.013059 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa8c47a-1561-4286-b4bb-cb5a884ae318 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:48.013188 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cdd098-f0f3-46bc-8aa9-41e9f311c996 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:48.030188 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 372/756] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:24:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:48.030188 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f9246-4130-4445-8196-e734583d4f48 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:48.030188 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f9246-4130-4445-8196-e734583d4f48 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:48.030188 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f9246-4130-4445-8196-e734583d4f48 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:48.030188 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 373/757] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:48.030188 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f9246-4130-4445-8196-e734583d4f48 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:48.138058 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cdd098-f0f3-46bc-8aa9-41e9f311c996 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:48.175992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f9246-4130-4445-8196-e734583d4f48 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:48.176525 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5037e18f-5a1e-4f3e-83d5-66f166c2e346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:48.177325 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5037e18f-5a1e-4f3e-83d5-66f166c2e346 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:48.177828 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5037e18f-5a1e-4f3e-83d5-66f166c2e346 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:48.178323 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5037e18f-5a1e-4f3e-83d5-66f166c2e346 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:48.179261 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5037e18f-5a1e-4f3e-83d5-66f166c2e346 None admin] RBAC: Adding query filter params (role.id=cb40cd8499f7418ca0da5ea0f1b87ff9, scope.project.id=59592706ae9442e08353158c625e9581, user.id=604380c191c8461bbb937b8d951238bc) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:48.179871 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5037e18f-5a1e-4f3e-83d5-66f166c2e346 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:48.181577 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5037e18f-5a1e-4f3e-83d5-66f166c2e346 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:48.196665 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 385/758] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:24:48 2022] GET /identity/v3/role_assignments?user.id=604380c191c8461bbb937b8d951238bc&scope.project.id=59592706ae9442e08353158c625e9581&role.id=cb40cd8499f7418ca0da5ea0f1b87ff9 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:24:48.198482 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb5841-1249-466b-aeec-1bfdfb31bbfd None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:48.199001 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb5841-1249-466b-aeec-1bfdfb31bbfd None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:48.199485 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb5841-1249-466b-aeec-1bfdfb31bbfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:48.227831 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb2523f-d5a8-4b26-b9f2-caf35f3d558a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:48.228827 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb2523f-d5a8-4b26-b9f2-caf35f3d558a None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:48.229116 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb2523f-d5a8-4b26-b9f2-caf35f3d558a None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:48.229393 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb2523f-d5a8-4b26-b9f2-caf35f3d558a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:48.238407 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-a7fb5841-1249-466b-aeec-1bfdfb31bbfd None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:48.239638 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-a7fb5841-1249-466b-aeec-1bfdfb31bbfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:48.247044 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb2523f-d5a8-4b26-b9f2-caf35f3d558a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:48.249113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb2523f-d5a8-4b26-b9f2-caf35f3d558a None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:48.286221 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-1bb2523f-d5a8-4b26-b9f2-caf35f3d558a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:48.286221 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:48.286221 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:48.286221 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:48.286221 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:48.286221 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:48.286221 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:48.286221 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:48.286221 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:48.286221 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:48.286221 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:48.286221 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:48.286221 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:48.286221 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:48.286221 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:48.286221 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:48.287643 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:48.287643 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:48.287643 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:48.287643 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:48.287643 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:48.287643 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:48.287643 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:48.287643 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:48.287643 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:48.287643 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:48.287643 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:48.287643 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:48.287643 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:48.287643 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:48.287643 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:48.287643 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:48.287643 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:48.289030 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:48.289030 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:48.289030 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:48.289030 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 374/759] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:48.293310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb2523f-d5a8-4b26-b9f2-caf35f3d558a None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:48.323631 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:48.349752 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 386/760] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:24:48.357351 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:48.472008 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb2523f-d5a8-4b26-b9f2-caf35f3d558a None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:48.515374 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6428d3a4-16c6-4842-9d8a-28304e2387e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:48.515979 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6428d3a4-16c6-4842-9d8a-28304e2387e5 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:48.516252 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6428d3a4-16c6-4842-9d8a-28304e2387e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:48.516549 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6428d3a4-16c6-4842-9d8a-28304e2387e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:48.517196 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6428d3a4-16c6-4842-9d8a-28304e2387e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:48.517519 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6428d3a4-16c6-4842-9d8a-28304e2387e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:48.518842 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6428d3a4-16c6-4842-9d8a-28304e2387e5 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:48.535674 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 375/761] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:24:48.541321 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb5841-1249-466b-aeec-1bfdfb31bbfd None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:48.657333 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb5841-1249-466b-aeec-1bfdfb31bbfd None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:48.700368 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cddf4d-c63b-4d38-9056-65f464975320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:48.701039 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cddf4d-c63b-4d38-9056-65f464975320 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:48.701394 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cddf4d-c63b-4d38-9056-65f464975320 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:48.702191 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cddf4d-c63b-4d38-9056-65f464975320 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:48.710144 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cddf4d-c63b-4d38-9056-65f464975320 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:48.711599 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cddf4d-c63b-4d38-9056-65f464975320 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:48.724158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-d1cddf4d-c63b-4d38-9056-65f464975320 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:24:48.724158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:48.724158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:48.724158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:48.724158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:48.724158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:48.724158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:48.724158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:48.724158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:48.724158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:48.724158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:48.724158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:48.724158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:24:48.724158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:24:48.724158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:24:48.724158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:24:48.725212 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:48.725212 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:48.725212 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:24:48.725212 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:48.725212 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:24:48.725212 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:48.725212 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:48.725212 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:48.725212 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:48.725212 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:48.725212 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:48.725212 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:48.725212 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:48.725212 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:48.725212 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:48.725212 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:48.725212 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:48.725212 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:48.726154 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:48.726154 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:48.726154 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:24:48.726154 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:24:48.726154 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:24:48.726154 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:24:48.726154 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:24:48.726154 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:24:48.726154 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:24:48.726154 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:48.726154 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 387/762] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Aug 24 05:24:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:48.732566 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6428d3a4-16c6-4842-9d8a-28304e2387e5 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:48.795507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cddf4d-c63b-4d38-9056-65f464975320 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:48.852771 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:48.860534 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:48.890121 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6428d3a4-16c6-4842-9d8a-28304e2387e5 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:48.917830 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cddf4d-c63b-4d38-9056-65f464975320 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:48.942735 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709532b-c26a-4343-af1f-d80fee682d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:48.943895 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709532b-c26a-4343-af1f-d80fee682d84 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:48.944737 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709532b-c26a-4343-af1f-d80fee682d84 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:48.945515 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709532b-c26a-4343-af1f-d80fee682d84 None admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:48.946720 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7709532b-c26a-4343-af1f-d80fee682d84 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:48.947628 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7709532b-c26a-4343-af1f-d80fee682d84 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:48.949866 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7709532b-c26a-4343-af1f-d80fee682d84 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:48.960338 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9def7fb-9f76-4abd-a21d-0843d5cbdd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:48.961010 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9def7fb-9f76-4abd-a21d-0843d5cbdd1f None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:48.961288 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9def7fb-9f76-4abd-a21d-0843d5cbdd1f None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:48.961653 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9def7fb-9f76-4abd-a21d-0843d5cbdd1f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:48.962152 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9def7fb-9f76-4abd-a21d-0843d5cbdd1f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:48.963546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9def7fb-9f76-4abd-a21d-0843d5cbdd1f None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:48.975286 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-f9def7fb-9f76-4abd-a21d-0843d5cbdd1f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 05:24:48.975286 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:48.975286 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:24:48.975286 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:24:48.975286 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:24:48.975286 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:24:48.975286 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:24:48.975286 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:24:48.975286 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 05:24:48.975286 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:48.975286 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:24:48.975286 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:48.975286 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:48.975286 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:48.975286 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:48.975286 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:48.975286 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:48.976914 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:48.976914 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:48.976914 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:48.976914 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:48.976914 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:48.976914 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:48.976914 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:24:48.976914 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:24:48.976914 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:24:48.976914 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:24:48.976914 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:48.976914 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:48.976914 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:48.976914 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:48.976914 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:48.976914 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:48.978291 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:48.978291 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:48.978291 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:48.978291 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:48.978291 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:48.978291 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:48.978291 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:48.978291 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:48.978291 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:48.978291 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:48.978291 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:24:48.978291 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:24:48.978291 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 05:24:48.978291 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:48.978291 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 388/763] 10.222.0.15 () {56 vars in 1129 bytes} [Wed Aug 24 05:24:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:48.979579 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243ce78-5ac1-49cb-8e29-439ccaee8281 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:48.979579 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243ce78-5ac1-49cb-8e29-439ccaee8281 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:48.979579 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243ce78-5ac1-49cb-8e29-439ccaee8281 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:48.984819 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 389/764] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:48.985776 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f243ce78-5ac1-49cb-8e29-439ccaee8281 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:48.989154 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 376/765] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Aug 24 05:24:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:48.990733 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c9de6-da99-4b30-80b9-9f8e2d4d877f None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:48.991235 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c9de6-da99-4b30-80b9-9f8e2d4d877f None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:48.991782 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c9de6-da99-4b30-80b9-9f8e2d4d877f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:49.016806 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-a03c9de6-da99-4b30-80b9-9f8e2d4d877f None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:49.019202 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-a03c9de6-da99-4b30-80b9-9f8e2d4d877f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:49.111195 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f243ce78-5ac1-49cb-8e29-439ccaee8281 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:49.120469 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 377/766] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:24:49.121577 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c9de6-da99-4b30-80b9-9f8e2d4d877f None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:49.128943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:49.197159 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:49.205200 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f9a54-d08e-4b7b-9eb4-cc4040fffd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:49.205836 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f9a54-d08e-4b7b-9eb4-cc4040fffd19 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:49.206122 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f9a54-d08e-4b7b-9eb4-cc4040fffd19 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:49.206749 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f9a54-d08e-4b7b-9eb4-cc4040fffd19 None admin] PATH_INFO: `/v3/services` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:49.207028 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9f9a54-d08e-4b7b-9eb4-cc4040fffd19 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:49.207418 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9f9a54-d08e-4b7b-9eb4-cc4040fffd19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:49.208732 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9f9a54-d08e-4b7b-9eb4-cc4040fffd19 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:49.230413 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 390/767] 10.222.0.15 () {56 vars in 1124 bytes} [Wed Aug 24 05:24:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:49.238648 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c9de6-da99-4b30-80b9-9f8e2d4d877f None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:49.241791 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f9a54-d08e-4b7b-9eb4-cc4040fffd19 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:49.278702 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5cbc66-88e0-4fb5-82aa-c9507edc1c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:49.279777 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5cbc66-88e0-4fb5-82aa-c9507edc1c58 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:49.280494 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5cbc66-88e0-4fb5-82aa-c9507edc1c58 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:49.281238 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5cbc66-88e0-4fb5-82aa-c9507edc1c58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:49.292573 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5cbc66-88e0-4fb5-82aa-c9507edc1c58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:49.294981 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5cbc66-88e0-4fb5-82aa-c9507edc1c58 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:49.312864 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-bf5cbc66-88e0-4fb5-82aa-c9507edc1c58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:49.312864 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:49.312864 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:49.312864 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:49.312864 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:49.312864 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:49.312864 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:49.312864 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:49.312864 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:49.312864 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:49.312864 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:49.312864 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:49.312864 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:49.312864 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:49.312864 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:49.312864 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:49.313860 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:49.313860 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:49.313860 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:49.313860 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:49.313860 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:49.313860 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:49.313860 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:49.313860 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:49.313860 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:49.313860 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:49.313860 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:49.313860 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:49.313860 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:49.313860 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:49.313860 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:49.313860 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:49.313860 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:49.314952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:49.314952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:49.314952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:49.317324 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 378/768] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:49.325627 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5cbc66-88e0-4fb5-82aa-c9507edc1c58 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:49.461184 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f9a54-d08e-4b7b-9eb4-cc4040fffd19 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:49.498859 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5188f2e8-f004-41b7-95a2-72605cd53f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:49.499890 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5188f2e8-f004-41b7-95a2-72605cd53f10 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:49.500456 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5188f2e8-f004-41b7-95a2-72605cd53f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:49.500967 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5188f2e8-f004-41b7-95a2-72605cd53f10 None admin] PATH_INFO: `/v3/services` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:49.501826 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5188f2e8-f004-41b7-95a2-72605cd53f10 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:49.502339 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5188f2e8-f004-41b7-95a2-72605cd53f10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:49.504092 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5188f2e8-f004-41b7-95a2-72605cd53f10 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:49.504529 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5cbc66-88e0-4fb5-82aa-c9507edc1c58 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:49.511467 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 391/769] 10.222.0.15 () {56 vars in 1124 bytes} [Wed Aug 24 05:24:49 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:49.517383 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5188f2e8-f004-41b7-95a2-72605cd53f10 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:49.535226 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:49.545702 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e414f04e-0c6b-4f29-aba3-c82ec7045e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:49.546459 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414f04e-0c6b-4f29-aba3-c82ec7045e70 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:49.546836 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414f04e-0c6b-4f29-aba3-c82ec7045e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:49.547203 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414f04e-0c6b-4f29-aba3-c82ec7045e70 None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:49.547952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e414f04e-0c6b-4f29-aba3-c82ec7045e70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:49.548392 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e414f04e-0c6b-4f29-aba3-c82ec7045e70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:49.550126 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e414f04e-0c6b-4f29-aba3-c82ec7045e70 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:49.569720 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 379/770] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:49.570787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:49.572576 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e414f04e-0c6b-4f29-aba3-c82ec7045e70 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:49.643034 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5188f2e8-f004-41b7-95a2-72605cd53f10 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:49.679138 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e414f04e-0c6b-4f29-aba3-c82ec7045e70 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:49.680192 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a9fcf-008e-4fce-93ea-34c5d9964991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:49.680900 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a9fcf-008e-4fce-93ea-34c5d9964991 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:49.681188 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a9fcf-008e-4fce-93ea-34c5d9964991 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:49.681467 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a9fcf-008e-4fce-93ea-34c5d9964991 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:49.682175 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a9fcf-008e-4fce-93ea-34c5d9964991 None admin] RBAC: Adding query filter params (interface=public, service_id=5c81033c267443b7a2de5d1611e115f9, region_id=RegionOne) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:49.682488 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a9fcf-008e-4fce-93ea-34c5d9964991 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:49.683868 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a9fcf-008e-4fce-93ea-34c5d9964991 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:49.693593 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 392/771] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Aug 24 05:24:49 2022] GET /identity/v3/endpoints?service_id=5c81033c267443b7a2de5d1611e115f9&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:49.694475 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a9fcf-008e-4fce-93ea-34c5d9964991 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:49.719183 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ed674-efa5-4d76-bdaf-ea48462486e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:49.720189 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ed674-efa5-4d76-bdaf-ea48462486e8 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:49.720806 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ed674-efa5-4d76-bdaf-ea48462486e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:49.721282 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ed674-efa5-4d76-bdaf-ea48462486e8 None admin] PATH_INFO: `/v3/services/network` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:49.722166 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ed674-efa5-4d76-bdaf-ea48462486e8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:49.723731 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ed674-efa5-4d76-bdaf-ea48462486e8 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:49.733562 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-ac2ed674-efa5-4d76-bdaf-ea48462486e8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 05:24:49.733562 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:49.733562 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:24:49.733562 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:24:49.733562 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:24:49.733562 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:24:49.733562 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:24:49.733562 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:24:49.733562 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 05:24:49.733562 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:49.733562 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:24:49.733562 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:49.733562 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:49.733562 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:49.733562 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:49.733562 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:49.733562 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:49.734548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:49.734548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:49.734548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:49.734548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:49.734548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:49.734548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:49.734548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:24:49.734548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:24:49.734548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:24:49.734548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:24:49.734548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:49.734548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:49.734548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:49.734548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:49.734548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:49.734548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:49.735397 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:49.735397 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:49.735397 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:49.735397 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:49.735397 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:49.735397 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:49.735397 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:49.735397 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:49.735397 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:49.735397 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:49.735397 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:24:49.735397 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:24:49.735397 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 05:24:49.735397 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:49.738268 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 380/772] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:24:49 2022] GET /identity/v3/services/network => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:49.739360 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ed674-efa5-4d76-bdaf-ea48462486e8 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:49.788754 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:49.803474 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:49.809472 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a9fcf-008e-4fce-93ea-34c5d9964991 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:49.851030 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ed674-efa5-4d76-bdaf-ea48462486e8 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:49.852026 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de237957-f10d-429c-b20a-cb24b4cd8ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:49.852712 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de237957-f10d-429c-b20a-cb24b4cd8ebd None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:49.852990 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de237957-f10d-429c-b20a-cb24b4cd8ebd None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:49.853261 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de237957-f10d-429c-b20a-cb24b4cd8ebd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:49.863346 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de237957-f10d-429c-b20a-cb24b4cd8ebd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:49.865257 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de237957-f10d-429c-b20a-cb24b4cd8ebd None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:49.878818 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-de237957-f10d-429c-b20a-cb24b4cd8ebd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:49.878818 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:49.878818 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:49.878818 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:49.878818 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:49.878818 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:49.878818 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:49.878818 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:49.878818 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:49.878818 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:49.878818 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:49.878818 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:49.878818 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:24:49.878818 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:24:49.878818 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:24:49.879751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:24:49.879751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:49.879751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:49.879751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:49.879751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:49.879751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:49.879751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:49.879751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:49.879751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:49.879751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:49.879751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:49.879751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:49.879751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:49.879751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:49.879751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:49.879751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:49.879751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:49.880708 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:24:49.880708 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:24:49.880708 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:24:49.880708 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:24:49.880708 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:24:49.880708 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:24:49.880708 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:49.880708 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:49.883086 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 393/773] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:24:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:49.884216 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de237957-f10d-429c-b20a-cb24b4cd8ebd None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:49.897761 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6f7fe-deb5-43a4-ad34-44822ec81ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:49.898770 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6f7fe-deb5-43a4-ad34-44822ec81ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:49.899349 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6f7fe-deb5-43a4-ad34-44822ec81ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:49.899825 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6f7fe-deb5-43a4-ad34-44822ec81ca4 None admin] PATH_INFO: `/v3/services` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:49.900681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f6f7fe-deb5-43a4-ad34-44822ec81ca4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:49.902421 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f6f7fe-deb5-43a4-ad34-44822ec81ca4 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:49.909728 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 381/774] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Aug 24 05:24:49 2022] GET /identity/v3/services => generated 772 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:49.911011 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6f7fe-deb5-43a4-ad34-44822ec81ca4 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:50.019328 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de237957-f10d-429c-b20a-cb24b4cd8ebd None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:50.038809 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6f7fe-deb5-43a4-ad34-44822ec81ca4 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:50.047681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:50.060131 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d226a7-2b60-40d7-9548-1afaf68f9dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:50.061233 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d226a7-2b60-40d7-9548-1afaf68f9dad None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:50.061718 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d226a7-2b60-40d7-9548-1afaf68f9dad None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:50.062227 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d226a7-2b60-40d7-9548-1afaf68f9dad None admin] PATH_INFO: `/v3/services` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:50.063831 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d226a7-2b60-40d7-9548-1afaf68f9dad None admin] RBAC: Adding query filter params (name=network) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:50.064366 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d226a7-2b60-40d7-9548-1afaf68f9dad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:50.066085 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d226a7-2b60-40d7-9548-1afaf68f9dad None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:50.071727 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 394/775] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Aug 24 05:24:49 2022] GET /identity/v3/services?name=network => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:50.073300 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e0738-062d-4b1a-b0f3-9ac45c48a6d5 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:50.073744 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e0738-062d-4b1a-b0f3-9ac45c48a6d5 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:50.074186 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e0738-062d-4b1a-b0f3-9ac45c48a6d5 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:50.075704 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 395/776] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:50.080923 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde6c73-e8dd-444d-81a2-7a7893fd8505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:50.081763 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde6c73-e8dd-444d-81a2-7a7893fd8505 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:50.082119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde6c73-e8dd-444d-81a2-7a7893fd8505 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:50.082472 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde6c73-e8dd-444d-81a2-7a7893fd8505 None admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:50.082943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8535fa-384d-4540-bd29-0597c0672726 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:50.083182 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cde6c73-e8dd-444d-81a2-7a7893fd8505 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:50.083496 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cde6c73-e8dd-444d-81a2-7a7893fd8505 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:50.083787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8535fa-384d-4540-bd29-0597c0672726 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:50.084242 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8535fa-384d-4540-bd29-0597c0672726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:50.085002 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cde6c73-e8dd-444d-81a2-7a7893fd8505 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:50.097841 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 382/777] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Aug 24 05:24:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:50.098820 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde6c73-e8dd-444d-81a2-7a7893fd8505 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:50.105502 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-2e8535fa-384d-4540-bd29-0597c0672726 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:50.107010 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-2e8535fa-384d-4540-bd29-0597c0672726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:50.123629 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:50.180775 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 396/778] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:24:50.192499 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8535fa-384d-4540-bd29-0597c0672726 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:50.249306 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde6c73-e8dd-444d-81a2-7a7893fd8505 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:50.285739 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8535fa-384d-4540-bd29-0597c0672726 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:50.320883 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875d9b19-feb6-4f92-9f1b-9d8eaa375db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:50.321561 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d9b19-feb6-4f92-9f1b-9d8eaa375db4 None admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:50.321848 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d9b19-feb6-4f92-9f1b-9d8eaa375db4 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:50.322128 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d9b19-feb6-4f92-9f1b-9d8eaa375db4 None admin] PATH_INFO: `/v3/projects/59592706ae9442e08353158c625e9581/users/fbecffb800584c5786813accd061421b/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:50.324065 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875d9b19-feb6-4f92-9f1b-9d8eaa375db4 None admin] RBAC: Authorizing `identity:create_grant(project_id=59592706ae9442e08353158c625e9581, user_id=fbecffb800584c5786813accd061421b, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:50.325564 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875d9b19-feb6-4f92-9f1b-9d8eaa375db4 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:50.339892 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a8f940-4555-4ad0-98e2-ffab352f20e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:50.340569 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8f940-4555-4ad0-98e2-ffab352f20e6 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:50.340867 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8f940-4555-4ad0-98e2-ffab352f20e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:50.341152 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8f940-4555-4ad0-98e2-ffab352f20e6 None admin] PATH_INFO: `/v3/services` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:50.356635 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a8f940-4555-4ad0-98e2-ffab352f20e6 None admin] RBAC: Adding query filter params (type=network) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:50.356635 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a8f940-4555-4ad0-98e2-ffab352f20e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:50.356635 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a8f940-4555-4ad0-98e2-ffab352f20e6 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:50.356635 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 383/779] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Aug 24 05:24:50 2022] GET /identity/v3/services?type=network => generated 350 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:50.356635 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a8f940-4555-4ad0-98e2-ffab352f20e6 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:50.390723 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:50.452823 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 397/780] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Aug 24 05:24:50 2022] PUT /identity/v3/projects/59592706ae9442e08353158c625e9581/users/fbecffb800584c5786813accd061421b/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:24:50.454445 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1cbf2-cf35-4ca6-8654-2f3e3765d6ec None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:50.454730 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1cbf2-cf35-4ca6-8654-2f3e3765d6ec None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:50.455010 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1cbf2-cf35-4ca6-8654-2f3e3765d6ec None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:50.456830 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 398/781] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:50.465193 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465db23e-fe22-4aae-ad7e-6a0c8f3c0256 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:50.465540 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465db23e-fe22-4aae-ad7e-6a0c8f3c0256 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:50.465757 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465db23e-fe22-4aae-ad7e-6a0c8f3c0256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:50.496466 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a8f940-4555-4ad0-98e2-ffab352f20e6 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:50.511581 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-465db23e-fe22-4aae-ad7e-6a0c8f3c0256 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:50.512938 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-465db23e-fe22-4aae-ad7e-6a0c8f3c0256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:50.518571 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:50.559782 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ebde2-8986-4be6-9e0e-71799f238d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:50.561383 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ebde2-8986-4be6-9e0e-71799f238d63 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:50.561942 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ebde2-8986-4be6-9e0e-71799f238d63 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:50.562421 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ebde2-8986-4be6-9e0e-71799f238d63 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:50.563364 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ebde2-8986-4be6-9e0e-71799f238d63 None admin] RBAC: Adding query filter params (interface=public, service_id=039602201f144ba5ad7fa934c08fde65, region_id=RegionOne) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:50.563955 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ebde2-8986-4be6-9e0e-71799f238d63 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:50.566376 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ebde2-8986-4be6-9e0e-71799f238d63 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:50.585288 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 384/782] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Aug 24 05:24:50 2022] GET /identity/v3/endpoints?service_id=039602201f144ba5ad7fa934c08fde65&interface=public®ion_id=RegionOne => generated 194 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:50.589319 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ebde2-8986-4be6-9e0e-71799f238d63 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:50.602593 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 399/783] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:24:50.609216 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465db23e-fe22-4aae-ad7e-6a0c8f3c0256 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:50.696440 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:50.704301 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ebde2-8986-4be6-9e0e-71799f238d63 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:50.727934 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465db23e-fe22-4aae-ad7e-6a0c8f3c0256 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:50.740842 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e8f3e2-1706-4be9-979c-b7c257f8fec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:50.741924 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8f3e2-1706-4be9-979c-b7c257f8fec4 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:50.742418 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8f3e2-1706-4be9-979c-b7c257f8fec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:50.742893 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8f3e2-1706-4be9-979c-b7c257f8fec4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:50.757063 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e8f3e2-1706-4be9-979c-b7c257f8fec4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:24:50.762244 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-98e8f3e2-1706-4be9-979c-b7c257f8fec4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:24:50.762244 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:50.762244 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:50.762244 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:50.762244 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:50.762244 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:50.762244 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:50.762244 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:50.762244 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:50.762244 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:50.762244 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:50.762244 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:50.762244 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:24:50.762244 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:24:50.762244 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:24:50.762244 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:24:50.763208 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:24:50.763208 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:24:50.763208 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:24:50.763208 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:24:50.763208 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:50.763208 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:50.763208 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:50.763208 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:50.763208 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:50.763208 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:50.763208 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:50.763208 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:50.763208 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:50.763208 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:50.763208 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:50.763208 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:50.764135 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:50.764135 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:50.764135 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:50.764135 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:50.764135 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:24:50.764135 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:24:50.764135 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:24:50.764135 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:24:50.764135 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:24:50.764135 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:24:50.764135 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:24:50.764135 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:50.766923 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 385/784] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Aug 24 05:24:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:50.770721 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba9128-f4f7-4859-8277-e08cf9b2e9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:50.771677 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba9128-f4f7-4859-8277-e08cf9b2e9ac None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:50.772814 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e8f3e2-1706-4be9-979c-b7c257f8fec4 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:50.776501 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba9128-f4f7-4859-8277-e08cf9b2e9ac None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:50.777955 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba9128-f4f7-4859-8277-e08cf9b2e9ac None admin] PATH_INFO: `/v3/services` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:50.778785 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eba9128-f4f7-4859-8277-e08cf9b2e9ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:50.786037 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eba9128-f4f7-4859-8277-e08cf9b2e9ac None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:50.796146 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 400/785] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Aug 24 05:24:50 2022] GET /identity/v3/services => generated 772 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:50.926655 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e8f3e2-1706-4be9-979c-b7c257f8fec4 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:50.971821 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a6f6a5-78ae-4369-91fa-4f530d5f01f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:50.972789 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6f6a5-78ae-4369-91fa-4f530d5f01f6 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:50.973135 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6f6a5-78ae-4369-91fa-4f530d5f01f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:50.973452 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6f6a5-78ae-4369-91fa-4f530d5f01f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:50.974281 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a6f6a5-78ae-4369-91fa-4f530d5f01f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:50.974827 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a6f6a5-78ae-4369-91fa-4f530d5f01f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:50.977017 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a6f6a5-78ae-4369-91fa-4f530d5f01f6 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:50.977254 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba9128-f4f7-4859-8277-e08cf9b2e9ac None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:50.983567 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:50.992816 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 386/786] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:24:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:24:50.999891 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a6f6a5-78ae-4369-91fa-4f530d5f01f6 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:51.047540 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:51.140911 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba9128-f4f7-4859-8277-e08cf9b2e9ac None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:51.166954 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a6f6a5-78ae-4369-91fa-4f530d5f01f6 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:51.188038 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f6e210-d9fa-4d24-a604-72782c9cdc92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:51.189528 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6e210-d9fa-4d24-a604-72782c9cdc92 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:51.189960 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6e210-d9fa-4d24-a604-72782c9cdc92 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:51.190392 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6e210-d9fa-4d24-a604-72782c9cdc92 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:51.203200 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f6e210-d9fa-4d24-a604-72782c9cdc92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:24:51.208531 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-97f6e210-d9fa-4d24-a604-72782c9cdc92 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:24:51.208531 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:51.208531 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:51.208531 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:51.208531 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:51.208531 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:51.208531 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:51.208531 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:51.208531 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:51.208531 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:51.208531 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:51.208531 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:51.208531 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:24:51.208531 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:24:51.208531 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:24:51.208531 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:24:51.209580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:24:51.209580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:24:51.209580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:24:51.209580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:24:51.209580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:51.209580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:51.209580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:51.209580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:51.209580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:51.209580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:51.209580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:51.209580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:51.209580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:51.209580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:51.209580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:51.209580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:51.210420 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:51.210420 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:51.210420 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:51.210420 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:51.210420 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:24:51.210420 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:24:51.210420 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:24:51.210420 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:24:51.210420 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:24:51.210420 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:24:51.210420 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:24:51.210420 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:51.211054 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 401/787] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Aug 24 05:24:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:24:51.216414 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f6e210-d9fa-4d24-a604-72782c9cdc92 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:51.222255 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa6b52e-622e-48be-8874-64320c179995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:51.227261 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6b52e-622e-48be-8874-64320c179995 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:51.227817 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6b52e-622e-48be-8874-64320c179995 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:51.228271 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6b52e-622e-48be-8874-64320c179995 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:51.247004 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa6b52e-622e-48be-8874-64320c179995 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:51.249581 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa6b52e-622e-48be-8874-64320c179995 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:51.270338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-9fa6b52e-622e-48be-8874-64320c179995 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:51.270338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:51.270338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:51.270338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:51.270338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:51.270338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:51.270338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:51.270338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:51.270338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:51.270338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:51.270338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:51.270338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:51.270338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:51.270338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:51.270338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:51.270338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:51.271338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:51.271338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:51.271338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:51.271338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:51.271338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:51.271338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:51.271338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:51.271338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:51.271338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:51.271338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:51.271338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:51.271338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:51.271338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:51.271338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:51.271338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:51.271338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:51.271338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:51.272246 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:51.272246 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:51.272246 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:51.274369 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 387/788] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:51.280780 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa6b52e-622e-48be-8874-64320c179995 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:51.348767 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:51.391542 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:51.407122 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f6e210-d9fa-4d24-a604-72782c9cdc92 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:51.450915 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa6b52e-622e-48be-8874-64320c179995 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:51.468596 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a104d4b-0ecd-401c-9697-11a9a9941c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:51.469212 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a104d4b-0ecd-401c-9697-11a9a9941c4f None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:51.469489 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a104d4b-0ecd-401c-9697-11a9a9941c4f None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:51.469764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a104d4b-0ecd-401c-9697-11a9a9941c4f None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:51.470425 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a104d4b-0ecd-401c-9697-11a9a9941c4f None admin] RBAC: Adding query filter params (name=service) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:51.470725 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a104d4b-0ecd-401c-9697-11a9a9941c4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:51.472320 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a104d4b-0ecd-401c-9697-11a9a9941c4f None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:51.488714 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 402/789] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Aug 24 05:24:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:51.491949 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a104d4b-0ecd-401c-9697-11a9a9941c4f None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:51.528425 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54978b6-fe5c-402f-9fff-13f14e1aa1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:51.528425 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54978b6-fe5c-402f-9fff-13f14e1aa1d3 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:51.528425 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54978b6-fe5c-402f-9fff-13f14e1aa1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:51.528425 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54978b6-fe5c-402f-9fff-13f14e1aa1d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:51.528851 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54978b6-fe5c-402f-9fff-13f14e1aa1d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:51.528851 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54978b6-fe5c-402f-9fff-13f14e1aa1d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:51.528851 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54978b6-fe5c-402f-9fff-13f14e1aa1d3 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:51.538848 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 388/790] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:24:51.544693 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54978b6-fe5c-402f-9fff-13f14e1aa1d3 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:51.662982 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54978b6-fe5c-402f-9fff-13f14e1aa1d3 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:51.695000 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:51.724694 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab4af2b-fc3e-49d7-8673-b7380942539c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:51.724694 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4af2b-fc3e-49d7-8673-b7380942539c None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:51.724694 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4af2b-fc3e-49d7-8673-b7380942539c None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:51.724694 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4af2b-fc3e-49d7-8673-b7380942539c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:51.725213 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab4af2b-fc3e-49d7-8673-b7380942539c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:51.725213 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab4af2b-fc3e-49d7-8673-b7380942539c None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:51.730871 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-3ab4af2b-fc3e-49d7-8673-b7380942539c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 05:24:51.730871 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:51.730871 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:51.730871 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:51.730871 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:51.730871 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:51.730871 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:51.730871 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:51.730871 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:51.730871 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:51.730871 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:51.730871 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:51.730871 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:24:51.730871 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:24:51.730871 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:24:51.730871 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:24:51.731768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:51.731768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:51.731768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:24:51.731768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:51.731768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:24:51.731768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:51.731768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:51.731768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:51.731768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:51.731768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:51.731768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:51.731768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:51.731768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:51.731768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:51.731768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:51.731768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:51.731768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:51.731768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:51.732849 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:51.732849 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:51.732849 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:24:51.732849 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:24:51.732849 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:24:51.732849 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:24:51.732849 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:24:51.732849 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:24:51.732849 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 05:24:51.732849 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:51.735925 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 389/791] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Aug 24 05:24:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:24:51.741698 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab4af2b-fc3e-49d7-8673-b7380942539c None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:51.790488 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a104d4b-0ecd-401c-9697-11a9a9941c4f None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:51.858791 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab4af2b-fc3e-49d7-8673-b7380942539c None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:51.894156 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:51.898655 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5556021-a367-4dfe-b228-4515383b128c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:51.899526 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5556021-a367-4dfe-b228-4515383b128c None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:51.901371 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5556021-a367-4dfe-b228-4515383b128c None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:51.907828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5556021-a367-4dfe-b228-4515383b128c None admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:51.907828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5556021-a367-4dfe-b228-4515383b128c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:51.907828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5556021-a367-4dfe-b228-4515383b128c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:51.907828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5556021-a367-4dfe-b228-4515383b128c None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:51.917520 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c6e8db-cd8b-4f43-929e-4fda68c69cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:51.919923 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6e8db-cd8b-4f43-929e-4fda68c69cb3 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:51.920376 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6e8db-cd8b-4f43-929e-4fda68c69cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:51.920816 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6e8db-cd8b-4f43-929e-4fda68c69cb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:51.944685 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c6e8db-cd8b-4f43-929e-4fda68c69cb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:51.947625 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c6e8db-cd8b-4f43-929e-4fda68c69cb3 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:51.948657 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 390/792] 10.222.0.15 () {56 vars in 1135 bytes} [Wed Aug 24 05:24:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:51.965030 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-b3c6e8db-cd8b-4f43-929e-4fda68c69cb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:51.965030 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:51.965030 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:51.965030 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:51.965030 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:51.965030 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:51.965030 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:51.965030 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:51.965030 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:51.965030 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:51.965030 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:51.965030 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:51.965030 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:51.965030 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:51.965030 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:51.965030 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:51.966158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:51.966158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:51.966158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:51.966158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:51.966158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:51.966158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:51.966158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:51.966158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:51.966158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:51.966158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:51.966158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:51.966158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:51.966158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:51.966158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:51.966158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:51.966158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:51.966158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:51.967354 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:51.967354 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:51.967354 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:51.969781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5556021-a367-4dfe-b228-4515383b128c None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:51.981365 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 403/793] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 490 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:51.986790 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c6e8db-cd8b-4f43-929e-4fda68c69cb3 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:52.072799 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:52.106946 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5556021-a367-4dfe-b228-4515383b128c None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:52.144643 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4db8873-152f-4343-9caf-72380d745996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:52.145294 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db8873-152f-4343-9caf-72380d745996 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:52.145574 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db8873-152f-4343-9caf-72380d745996 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:52.145850 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db8873-152f-4343-9caf-72380d745996 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:52.158251 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4db8873-152f-4343-9caf-72380d745996 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:52.160009 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4db8873-152f-4343-9caf-72380d745996 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:52.175018 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-e4db8873-152f-4343-9caf-72380d745996 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:52.175018 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:52.175018 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:52.175018 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:52.175018 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:52.175018 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:52.175018 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:52.175018 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:52.175018 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:52.175018 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:52.175018 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:52.175018 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:52.175018 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:52.175018 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:52.175018 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:52.175018 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:52.176239 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:52.176239 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:52.176239 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:52.176239 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:52.176239 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:52.176239 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:52.176239 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:52.176239 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:52.176239 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:52.176239 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:52.176239 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:52.176239 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:52.176239 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:52.176239 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:52.176239 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:52.176239 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:52.176239 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:52.177152 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:52.177152 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:52.177152 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:52.177152 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 391/794] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:52.184942 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4db8873-152f-4343-9caf-72380d745996 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:52.229817 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c6e8db-cd8b-4f43-929e-4fda68c69cb3 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:52.277209 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc69429-b1a8-4a20-8534-fc0f52677d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:52.277784 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc69429-b1a8-4a20-8534-fc0f52677d4c None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:52.278320 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc69429-b1a8-4a20-8534-fc0f52677d4c None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:52.278600 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc69429-b1a8-4a20-8534-fc0f52677d4c None admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:52.280801 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc69429-b1a8-4a20-8534-fc0f52677d4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:52.281388 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc69429-b1a8-4a20-8534-fc0f52677d4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:52.283115 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc69429-b1a8-4a20-8534-fc0f52677d4c None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:52.296852 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:52.301550 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 404/795] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:52.302553 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4db8873-152f-4343-9caf-72380d745996 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:52.303140 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499689f-ffbb-4cb0-9afa-d9607eb05b69 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:52.303527 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499689f-ffbb-4cb0-9afa-d9607eb05b69 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:52.303893 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499689f-ffbb-4cb0-9afa-d9607eb05b69 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:52.305480 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 405/796] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:52.307419 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499689f-ffbb-4cb0-9afa-d9607eb05b69 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:52.350887 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c7599-fe7e-4bee-9112-eed57f0a2f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:52.351536 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c7599-fe7e-4bee-9112-eed57f0a2f1a None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:52.351819 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c7599-fe7e-4bee-9112-eed57f0a2f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:52.352094 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c7599-fe7e-4bee-9112-eed57f0a2f1a None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:52.352826 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4c7599-fe7e-4bee-9112-eed57f0a2f1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:52.353137 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4c7599-fe7e-4bee-9112-eed57f0a2f1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:52.354572 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4c7599-fe7e-4bee-9112-eed57f0a2f1a None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:52.356067 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:52.373711 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 392/797] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:52.375283 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04838735-bf91-44c8-a2c9-2c8bf0cc6c66 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:52.381185 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04838735-bf91-44c8-a2c9-2c8bf0cc6c66 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:52.381484 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04838735-bf91-44c8-a2c9-2c8bf0cc6c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:52.402593 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-04838735-bf91-44c8-a2c9-2c8bf0cc6c66 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:52.405157 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-04838735-bf91-44c8-a2c9-2c8bf0cc6c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:52.441529 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499689f-ffbb-4cb0-9afa-d9607eb05b69 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:52.487614 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef156f88-4d41-4e5d-b3fb-3d4c3412264b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:52.488712 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef156f88-4d41-4e5d-b3fb-3d4c3412264b None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:52.489139 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef156f88-4d41-4e5d-b3fb-3d4c3412264b None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:52.489555 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef156f88-4d41-4e5d-b3fb-3d4c3412264b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:52.490467 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 393/798] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:24:52.491459 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04838735-bf91-44c8-a2c9-2c8bf0cc6c66 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:52.497809 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef156f88-4d41-4e5d-b3fb-3d4c3412264b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:52.499978 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef156f88-4d41-4e5d-b3fb-3d4c3412264b None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:52.513080 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-ef156f88-4d41-4e5d-b3fb-3d4c3412264b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 05:24:52.513080 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:52.513080 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:52.513080 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:52.513080 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:52.513080 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:52.513080 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:52.513080 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:52.513080 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:52.513080 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:52.513080 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:52.513080 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:52.513080 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:24:52.513080 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:24:52.513080 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:24:52.513080 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:24:52.514295 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:52.514295 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:52.514295 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:24:52.514295 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:52.514295 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:24:52.514295 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:52.514295 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:52.514295 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:52.514295 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:52.514295 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:52.514295 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:52.514295 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:52.514295 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:52.514295 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:52.514295 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:52.514295 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:52.514295 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:52.514295 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:52.515359 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:52.515359 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:52.515359 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:24:52.515359 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:24:52.515359 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:24:52.515359 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:24:52.515359 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:24:52.515359 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:24:52.515359 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 05:24:52.515359 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:52.517940 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 406/799] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Aug 24 05:24:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:52.530064 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef156f88-4d41-4e5d-b3fb-3d4c3412264b None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:52.599327 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:52.614921 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04838735-bf91-44c8-a2c9-2c8bf0cc6c66 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:52.658456 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b338b77-1506-4ab7-a1ad-bd51ecb2365b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:52.659748 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b338b77-1506-4ab7-a1ad-bd51ecb2365b None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:52.660287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b338b77-1506-4ab7-a1ad-bd51ecb2365b None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:52.661360 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b338b77-1506-4ab7-a1ad-bd51ecb2365b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:52.680427 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b338b77-1506-4ab7-a1ad-bd51ecb2365b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:52.682114 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b338b77-1506-4ab7-a1ad-bd51ecb2365b None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:52.701032 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-4b338b77-1506-4ab7-a1ad-bd51ecb2365b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:52.701032 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:52.701032 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:52.701032 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:52.701032 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:52.701032 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:52.701032 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:52.701032 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:52.701032 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:52.701032 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:52.701032 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:52.701032 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:52.701032 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:24:52.701032 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:24:52.701032 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:24:52.702407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:24:52.702407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:52.702407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:52.702407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:52.702407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:52.702407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:52.702407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:52.702407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:52.702407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:52.702407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:52.702407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:52.702407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:52.702407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:52.702407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:52.702407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:52.702407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:52.702407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:52.703570 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:24:52.703570 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:24:52.703570 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:24:52.703570 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:24:52.703570 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:24:52.703570 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:24:52.703570 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:52.703570 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:52.706297 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 394/800] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:24:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:52.707707 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef8ccc-f718-4269-936f-c758c751e677 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:52.708177 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef8ccc-f718-4269-936f-c758c751e677 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:52.708653 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef8ccc-f718-4269-936f-c758c751e677 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:52.710313 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 395/801] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:52.711191 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ef8ccc-f718-4269-936f-c758c751e677 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:52.737025 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef156f88-4d41-4e5d-b3fb-3d4c3412264b None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:52.744743 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:52.776087 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2b306a-0a6a-474e-934a-8b4a6c6bb294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:52.776890 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b306a-0a6a-474e-934a-8b4a6c6bb294 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:52.777242 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b306a-0a6a-474e-934a-8b4a6c6bb294 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:52.777588 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b306a-0a6a-474e-934a-8b4a6c6bb294 None admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:52.778326 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2b306a-0a6a-474e-934a-8b4a6c6bb294 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:52.778701 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2b306a-0a6a-474e-934a-8b4a6c6bb294 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:52.780243 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2b306a-0a6a-474e-934a-8b4a6c6bb294 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:52.800933 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 407/802] 10.222.0.15 () {56 vars in 1149 bytes} [Wed Aug 24 05:24:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:52.800933 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892b2f0-47c9-4c6d-8a36-7e7d38e99463 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:52.800933 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892b2f0-47c9-4c6d-8a36-7e7d38e99463 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:52.800933 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892b2f0-47c9-4c6d-8a36-7e7d38e99463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:52.818288 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-d892b2f0-47c9-4c6d-8a36-7e7d38e99463 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:52.819481 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-d892b2f0-47c9-4c6d-8a36-7e7d38e99463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:52.840917 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ef8ccc-f718-4269-936f-c758c751e677 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:52.879513 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b9d675-fee0-4154-b1c7-5a629d9a0fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:52.880123 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9d675-fee0-4154-b1c7-5a629d9a0fa9 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:52.880476 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9d675-fee0-4154-b1c7-5a629d9a0fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:52.880760 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9d675-fee0-4154-b1c7-5a629d9a0fa9 None admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:52.881459 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b9d675-fee0-4154-b1c7-5a629d9a0fa9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:52.881768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b9d675-fee0-4154-b1c7-5a629d9a0fa9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:52.883208 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b9d675-fee0-4154-b1c7-5a629d9a0fa9 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:52.896271 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 396/803] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Aug 24 05:24:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:52.897166 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b9d675-fee0-4154-b1c7-5a629d9a0fa9 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:52.902742 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 408/804] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:24:52.904187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892b2f0-47c9-4c6d-8a36-7e7d38e99463 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:52.913422 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:53.018716 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:53.028954 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b9d675-fee0-4154-b1c7-5a629d9a0fa9 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:53.039012 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892b2f0-47c9-4c6d-8a36-7e7d38e99463 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:53.070635 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667b0fb-a625-4042-9bed-bd6a8ff3a9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:53.071676 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667b0fb-a625-4042-9bed-bd6a8ff3a9c6 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:53.072124 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667b0fb-a625-4042-9bed-bd6a8ff3a9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:53.072438 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667b0fb-a625-4042-9bed-bd6a8ff3a9c6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:53.073050 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8667b0fb-a625-4042-9bed-bd6a8ff3a9c6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:53.074565 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8667b0fb-a625-4042-9bed-bd6a8ff3a9c6 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:53.084279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3636dd70-598f-4921-b46e-d8dccfb517b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:53.085159 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636dd70-598f-4921-b46e-d8dccfb517b7 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:53.085647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636dd70-598f-4921-b46e-d8dccfb517b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:53.086067 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636dd70-598f-4921-b46e-d8dccfb517b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:53.088670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-8667b0fb-a625-4042-9bed-bd6a8ff3a9c6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 05:24:53.088670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:53.088670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:24:53.088670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:24:53.088670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:24:53.088670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:24:53.088670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:24:53.088670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:24:53.088670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 05:24:53.088670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:53.088670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:24:53.088670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:53.088670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:53.088670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:53.088670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:53.088670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:53.088670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:53.090289 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:53.090289 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:53.090289 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:53.090289 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:53.090289 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:53.090289 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:53.090289 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:24:53.090289 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:24:53.090289 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:24:53.090289 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:24:53.090289 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:53.090289 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:53.090289 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:53.090289 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:53.090289 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:53.090289 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:53.092930 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:53.092930 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:53.092930 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:53.092930 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:53.092930 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:53.092930 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:53.092930 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:53.092930 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:53.092930 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:53.092930 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:53.092930 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:24:53.092930 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:24:53.092930 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 05:24:53.092930 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:53.092930 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 397/805] 10.222.0.15 () {56 vars in 1129 bytes} [Wed Aug 24 05:24:52 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:53.096568 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667b0fb-a625-4042-9bed-bd6a8ff3a9c6 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:53.110498 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3636dd70-598f-4921-b46e-d8dccfb517b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:53.112069 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3636dd70-598f-4921-b46e-d8dccfb517b7 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:53.127847 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-3636dd70-598f-4921-b46e-d8dccfb517b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:53.127847 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:53.127847 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:53.127847 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:53.127847 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:53.127847 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:53.127847 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:53.127847 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:53.127847 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:53.127847 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:53.127847 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:53.127847 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:53.127847 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:53.127847 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:53.127847 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:53.127847 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:53.128887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:53.128887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:53.128887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:53.128887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:53.128887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:53.128887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:53.128887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:53.128887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:53.128887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:53.128887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:53.128887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:53.128887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:53.128887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:53.128887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:53.128887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:53.128887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:53.128887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:53.130059 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:53.130059 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:53.130059 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:53.134071 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 409/806] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:53.135027 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3636dd70-598f-4921-b46e-d8dccfb517b7 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:53.200573 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:53.241070 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667b0fb-a625-4042-9bed-bd6a8ff3a9c6 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:53.256200 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3636dd70-598f-4921-b46e-d8dccfb517b7 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:53.275161 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0bc80-d0e7-4b5d-9cf5-32b3f599c26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:53.275830 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0bc80-d0e7-4b5d-9cf5-32b3f599c26a None admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:53.276111 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0bc80-d0e7-4b5d-9cf5-32b3f599c26a None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:53.276389 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0bc80-d0e7-4b5d-9cf5-32b3f599c26a None admin] PATH_INFO: `/v3/projects/59592706ae9442e08353158c625e9581/users/bb2b6cae4a004539a95fc7dd558db255/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:53.278343 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb0bc80-d0e7-4b5d-9cf5-32b3f599c26a None admin] RBAC: Authorizing `identity:create_grant(project_id=59592706ae9442e08353158c625e9581, user_id=bb2b6cae4a004539a95fc7dd558db255, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:53.280358 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb0bc80-d0e7-4b5d-9cf5-32b3f599c26a None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:53.293423 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628859df-c7fa-40f0-8708-b47d75150bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:53.293971 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628859df-c7fa-40f0-8708-b47d75150bfd None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:53.294264 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628859df-c7fa-40f0-8708-b47d75150bfd None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:53.294543 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628859df-c7fa-40f0-8708-b47d75150bfd None admin] PATH_INFO: `/v3/services` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:53.296016 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628859df-c7fa-40f0-8708-b47d75150bfd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:53.296389 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628859df-c7fa-40f0-8708-b47d75150bfd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:53.297779 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628859df-c7fa-40f0-8708-b47d75150bfd None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:53.302373 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 410/807] 10.222.0.15 () {56 vars in 1124 bytes} [Wed Aug 24 05:24:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:53.317541 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628859df-c7fa-40f0-8708-b47d75150bfd None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:53.317541 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:53.401515 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628859df-c7fa-40f0-8708-b47d75150bfd None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:53.413050 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 398/808] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Aug 24 05:24:53 2022] PUT /identity/v3/projects/59592706ae9442e08353158c625e9581/users/bb2b6cae4a004539a95fc7dd558db255/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:24:53.413790 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0bc80-d0e7-4b5d-9cf5-32b3f599c26a None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:53.447361 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dae00f-05e3-4bac-8781-b81e301ce305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:53.447996 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dae00f-05e3-4bac-8781-b81e301ce305 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:53.448277 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dae00f-05e3-4bac-8781-b81e301ce305 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:53.448593 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dae00f-05e3-4bac-8781-b81e301ce305 None admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:53.449254 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dae00f-05e3-4bac-8781-b81e301ce305 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:53.449564 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dae00f-05e3-4bac-8781-b81e301ce305 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:53.451073 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dae00f-05e3-4bac-8781-b81e301ce305 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:53.469618 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 411/809] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:53.469618 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f97af94-4635-412e-9011-41b8a23112c5 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:53.469618 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f97af94-4635-412e-9011-41b8a23112c5 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:53.469618 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f97af94-4635-412e-9011-41b8a23112c5 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:53.469618 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 412/810] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:53.469618 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f97af94-4635-412e-9011-41b8a23112c5 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:53.540432 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0bc80-d0e7-4b5d-9cf5-32b3f599c26a None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:53.547470 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:53.581042 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:53.586202 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52350747-2b0b-4ba2-b351-f33c205761d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:53.586844 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52350747-2b0b-4ba2-b351-f33c205761d6 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:53.587137 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52350747-2b0b-4ba2-b351-f33c205761d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:53.587389 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52350747-2b0b-4ba2-b351-f33c205761d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:53.595062 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f97af94-4635-412e-9011-41b8a23112c5 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:53.599794 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52350747-2b0b-4ba2-b351-f33c205761d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:24:53.604557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-52350747-2b0b-4ba2-b351-f33c205761d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:24:53.604557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:53.604557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:53.604557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:53.604557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:53.604557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:53.604557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:53.604557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:53.604557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:53.604557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:53.604557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:53.604557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:53.604557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:24:53.604557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:24:53.604557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:24:53.604557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:24:53.605509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:24:53.605509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:24:53.605509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:24:53.605509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:24:53.605509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:53.605509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:53.605509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:53.605509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:53.605509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:53.605509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:53.605509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:53.605509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:53.605509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:53.605509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:53.605509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:53.605509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:53.606354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:53.606354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:53.606354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:53.606354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:53.606354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:24:53.606354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:24:53.606354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:24:53.606354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:24:53.606354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:24:53.606354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:24:53.606354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:24:53.606354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:53.606354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 399/811] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Aug 24 05:24:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:53.607115 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52350747-2b0b-4ba2-b351-f33c205761d6 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:53.639811 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f8bab3-c1b9-433f-a79d-fab07eb2082e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:53.640783 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8bab3-c1b9-433f-a79d-fab07eb2082e None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:53.641068 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8bab3-c1b9-433f-a79d-fab07eb2082e None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:53.641346 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8bab3-c1b9-433f-a79d-fab07eb2082e None admin] PATH_INFO: `/v3/services` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:53.641989 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f8bab3-c1b9-433f-a79d-fab07eb2082e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:53.642295 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f8bab3-c1b9-433f-a79d-fab07eb2082e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:53.643773 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f8bab3-c1b9-433f-a79d-fab07eb2082e None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:53.648889 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 413/812] 10.222.0.15 () {56 vars in 1124 bytes} [Wed Aug 24 05:24:53 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:53.650059 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae9b96-9a48-4193-8fc6-d33cc24317e5 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:53.650343 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae9b96-9a48-4193-8fc6-d33cc24317e5 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:53.650619 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae9b96-9a48-4193-8fc6-d33cc24317e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:53.669452 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-a2ae9b96-9a48-4193-8fc6-d33cc24317e5 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:53.675758 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-a2ae9b96-9a48-4193-8fc6-d33cc24317e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:53.721131 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52350747-2b0b-4ba2-b351-f33c205761d6 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:53.743577 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 414/813] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:24:53.744785 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae9b96-9a48-4193-8fc6-d33cc24317e5 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:53.769612 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd47d24-217e-48c4-b6ce-29a0dc9d005b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:53.771196 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd47d24-217e-48c4-b6ce-29a0dc9d005b None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:53.771960 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd47d24-217e-48c4-b6ce-29a0dc9d005b None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:53.772468 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd47d24-217e-48c4-b6ce-29a0dc9d005b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:53.783541 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd47d24-217e-48c4-b6ce-29a0dc9d005b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:53.786495 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd47d24-217e-48c4-b6ce-29a0dc9d005b None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:53.802364 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-ebd47d24-217e-48c4-b6ce-29a0dc9d005b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:53.802364 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:53.802364 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:53.802364 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:53.802364 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:53.802364 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:53.802364 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:53.802364 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:53.802364 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:53.802364 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:53.802364 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:53.802364 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:53.802364 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:24:53.802364 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:24:53.802364 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:24:53.803275 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:24:53.803275 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:53.803275 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:53.803275 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:53.803275 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:53.803275 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:53.803275 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:53.803275 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:53.803275 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:53.803275 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:53.803275 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:53.803275 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:53.803275 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:53.803275 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:53.803275 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:53.803275 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:53.803275 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:53.804171 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:24:53.804171 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:24:53.804171 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:24:53.804171 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:24:53.804171 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:24:53.804171 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:24:53.804171 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:53.804171 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:53.811561 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 400/814] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:24:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:53.811561 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd47d24-217e-48c4-b6ce-29a0dc9d005b None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:53.817000 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:53.876019 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:53.880585 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae9b96-9a48-4193-8fc6-d33cc24317e5 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:53.924776 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1485acb6-48a0-437b-976f-8a973c789588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:53.925297 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1485acb6-48a0-437b-976f-8a973c789588 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:53.925581 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1485acb6-48a0-437b-976f-8a973c789588 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:53.925856 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1485acb6-48a0-437b-976f-8a973c789588 None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:53.926526 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1485acb6-48a0-437b-976f-8a973c789588 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:53.927081 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1485acb6-48a0-437b-976f-8a973c789588 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:53.928209 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1485acb6-48a0-437b-976f-8a973c789588 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:53.934722 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd47d24-217e-48c4-b6ce-29a0dc9d005b None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:53.943081 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 415/815] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:24:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:53.944304 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1485acb6-48a0-437b-976f-8a973c789588 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:53.985495 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883cd3e-2b3b-4e6d-87e9-093b40b739ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:53.986490 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883cd3e-2b3b-4e6d-87e9-093b40b739ec None admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:53.986975 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883cd3e-2b3b-4e6d-87e9-093b40b739ec None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:53.987510 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883cd3e-2b3b-4e6d-87e9-093b40b739ec None admin] PATH_INFO: `/v3/endpoints` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:53.988426 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6883cd3e-2b3b-4e6d-87e9-093b40b739ec None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:53.990553 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6883cd3e-2b3b-4e6d-87e9-093b40b739ec None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:54.054868 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1485acb6-48a0-437b-976f-8a973c789588 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:54.072252 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:54.089022 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05698d64-5a97-404f-8547-0e31b292c755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:54.089628 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05698d64-5a97-404f-8547-0e31b292c755 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:54.089908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05698d64-5a97-404f-8547-0e31b292c755 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:54.090181 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05698d64-5a97-404f-8547-0e31b292c755 None admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:54.090872 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05698d64-5a97-404f-8547-0e31b292c755 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:54.091183 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05698d64-5a97-404f-8547-0e31b292c755 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:54.093217 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05698d64-5a97-404f-8547-0e31b292c755 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:54.111461 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 416/816] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Aug 24 05:24:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:54.112231 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05698d64-5a97-404f-8547-0e31b292c755 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:54.121078 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 401/817] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:24:53 2022] POST /identity/v3/endpoints => generated 346 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:24:54.122105 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883cd3e-2b3b-4e6d-87e9-093b40b739ec None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:54.261288 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883cd3e-2b3b-4e6d-87e9-093b40b739ec None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:54.264479 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05698d64-5a97-404f-8547-0e31b292c755 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:54.265691 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:54.304832 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f9ef67-9b06-4d46-a2ed-218f2b78c176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:54.305713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9ef67-9b06-4d46-a2ed-218f2b78c176 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:54.306008 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9ef67-9b06-4d46-a2ed-218f2b78c176 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:54.307201 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9ef67-9b06-4d46-a2ed-218f2b78c176 None admin] PATH_INFO: `/v3/services/network` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:54.307805 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f9ef67-9b06-4d46-a2ed-218f2b78c176 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:54.309256 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f9ef67-9b06-4d46-a2ed-218f2b78c176 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:54.312822 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a80378e-1c32-4f10-a1ce-99176a300134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:54.315291 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a80378e-1c32-4f10-a1ce-99176a300134 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:54.315291 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a80378e-1c32-4f10-a1ce-99176a300134 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:54.315291 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a80378e-1c32-4f10-a1ce-99176a300134 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:54.320159 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-56f9ef67-9b06-4d46-a2ed-218f2b78c176 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 05:24:54.320159 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:54.320159 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:24:54.320159 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:24:54.320159 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:24:54.320159 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:24:54.320159 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:24:54.320159 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:24:54.320159 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 05:24:54.320159 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:54.320159 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:24:54.320159 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:54.320159 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:54.320159 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:54.320159 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:54.320159 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:54.320159 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:54.321204 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:54.321204 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:54.321204 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:54.321204 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:54.321204 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:54.321204 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:54.321204 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:24:54.321204 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:24:54.321204 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:24:54.321204 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:24:54.321204 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:54.321204 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:54.321204 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:54.321204 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:54.321204 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:54.321204 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:54.322060 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:54.322060 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:54.322060 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:54.322060 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:54.322060 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:54.322060 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:54.322060 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:54.322060 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:54.322060 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:54.322060 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:54.322060 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:24:54.322060 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:24:54.322060 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 05:24:54.322060 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:54.322060 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 402/818] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:24:54 2022] GET /identity/v3/services/network => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:54.322868 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f9ef67-9b06-4d46-a2ed-218f2b78c176 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:54.332417 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a80378e-1c32-4f10-a1ce-99176a300134 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:54.334674 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a80378e-1c32-4f10-a1ce-99176a300134 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:54.353627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-4a80378e-1c32-4f10-a1ce-99176a300134 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:54.353627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:54.353627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:54.353627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:54.353627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:54.353627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:54.353627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:54.353627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:54.353627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:54.353627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:54.353627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:54.353627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:54.353627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:54.353627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:54.353627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:54.353627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:54.354635 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:54.354635 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:54.354635 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:54.354635 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:54.354635 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:54.354635 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:54.354635 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:54.354635 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:54.354635 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:54.354635 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:54.354635 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:54.354635 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:54.354635 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:54.354635 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:54.354635 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:54.354635 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:54.354635 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:54.355533 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:54.355533 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:54.355533 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:54.364422 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 417/819] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:54.366125 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a80378e-1c32-4f10-a1ce-99176a300134 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:54.416463 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:54.449789 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f9ef67-9b06-4d46-a2ed-218f2b78c176 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:54.487047 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d356737d-e1b3-40af-82de-e15befd7d7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:54.487817 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d356737d-e1b3-40af-82de-e15befd7d7a8 None admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:54.488167 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d356737d-e1b3-40af-82de-e15befd7d7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:54.488543 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d356737d-e1b3-40af-82de-e15befd7d7a8 None admin] PATH_INFO: `/v3/projects/59592706ae9442e08353158c625e9581/users/604380c191c8461bbb937b8d951238bc/roles/cb40cd8499f7418ca0da5ea0f1b87ff9` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:54.495029 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a80378e-1c32-4f10-a1ce-99176a300134 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:54.506814 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d356737d-e1b3-40af-82de-e15befd7d7a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=59592706ae9442e08353158c625e9581, user_id=604380c191c8461bbb937b8d951238bc, role_id=cb40cd8499f7418ca0da5ea0f1b87ff9)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:54.509015 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d356737d-e1b3-40af-82de-e15befd7d7a8 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:54.534559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7875c4-1404-454b-be29-3fc32a8b92c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:54.535269 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7875c4-1404-454b-be29-3fc32a8b92c9 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:54.535624 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7875c4-1404-454b-be29-3fc32a8b92c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:54.535971 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7875c4-1404-454b-be29-3fc32a8b92c9 None admin] PATH_INFO: `/v3/services` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:54.536729 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7875c4-1404-454b-be29-3fc32a8b92c9 None admin] RBAC: Adding query filter params (name=network) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:54.537104 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7875c4-1404-454b-be29-3fc32a8b92c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:54.538688 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7875c4-1404-454b-be29-3fc32a8b92c9 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:54.543468 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 418/820] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Aug 24 05:24:54 2022] GET /identity/v3/services?name=network => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:54.549114 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7875c4-1404-454b-be29-3fc32a8b92c9 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:54.648762 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7875c4-1404-454b-be29-3fc32a8b92c9 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:54.684103 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f65b0c-63dd-4e1a-a5dd-26989df6e586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:54.684820 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f65b0c-63dd-4e1a-a5dd-26989df6e586 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:54.685102 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f65b0c-63dd-4e1a-a5dd-26989df6e586 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:54.685377 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f65b0c-63dd-4e1a-a5dd-26989df6e586 None admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:54.686560 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f65b0c-63dd-4e1a-a5dd-26989df6e586 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:54.686866 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f65b0c-63dd-4e1a-a5dd-26989df6e586 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:54.688224 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f65b0c-63dd-4e1a-a5dd-26989df6e586 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:54.704444 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 419/821] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:54.704444 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f65b0c-63dd-4e1a-a5dd-26989df6e586 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:54.715013 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:54.808744 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 403/822] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Aug 24 05:24:54 2022] PUT /identity/v3/projects/59592706ae9442e08353158c625e9581/users/604380c191c8461bbb937b8d951238bc/roles/cb40cd8499f7418ca0da5ea0f1b87ff9 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:24:54.810243 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d356737d-e1b3-40af-82de-e15befd7d7a8 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:54.827197 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f65b0c-63dd-4e1a-a5dd-26989df6e586 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:54.870312 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:54.914378 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c67c14f-ad6f-4074-b643-0db96f559256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:54.915420 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67c14f-ad6f-4074-b643-0db96f559256 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:54.915924 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67c14f-ad6f-4074-b643-0db96f559256 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:54.916427 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67c14f-ad6f-4074-b643-0db96f559256 None admin] PATH_INFO: `/v3/services` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:54.917290 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c67c14f-ad6f-4074-b643-0db96f559256 None admin] RBAC: Adding query filter params (type=network) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:54.917914 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c67c14f-ad6f-4074-b643-0db96f559256 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:54.919557 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c67c14f-ad6f-4074-b643-0db96f559256 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:54.925927 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 420/823] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Aug 24 05:24:54 2022] GET /identity/v3/services?type=network => generated 350 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:54.933356 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c67c14f-ad6f-4074-b643-0db96f559256 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:54.989749 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:55.013890 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d356737d-e1b3-40af-82de-e15befd7d7a8 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:55.063697 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b2cf6a-edf4-49e5-a5c4-421c7d5e136d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:55.065065 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b2cf6a-edf4-49e5-a5c4-421c7d5e136d None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:55.065569 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b2cf6a-edf4-49e5-a5c4-421c7d5e136d None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:55.066064 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b2cf6a-edf4-49e5-a5c4-421c7d5e136d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:55.075856 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b2cf6a-edf4-49e5-a5c4-421c7d5e136d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:55.078864 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b2cf6a-edf4-49e5-a5c4-421c7d5e136d None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:55.095691 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-b7b2cf6a-edf4-49e5-a5c4-421c7d5e136d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:24:55.095691 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:55.095691 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:55.095691 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:55.095691 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:55.095691 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:55.095691 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:55.095691 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:55.095691 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:55.095691 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:55.095691 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:55.095691 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:55.095691 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:24:55.095691 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:24:55.095691 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:24:55.095691 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:24:55.096878 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:55.096878 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:55.096878 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:24:55.096878 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:55.096878 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:24:55.096878 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:55.096878 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:55.096878 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:55.096878 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:55.096878 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:55.096878 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:55.096878 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:55.096878 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:55.096878 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:55.096878 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:55.096878 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:55.096878 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:55.096878 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:55.098494 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:55.098494 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:55.098494 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:24:55.098494 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:24:55.098494 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:24:55.098494 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:24:55.098494 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:24:55.098494 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:24:55.098494 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:24:55.098494 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:55.098494 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 404/824] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Aug 24 05:24:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:55.103407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b2cf6a-edf4-49e5-a5c4-421c7d5e136d None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:55.114104 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c67c14f-ad6f-4074-b643-0db96f559256 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:55.159230 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c71d73-1a24-43f2-b604-3d5accb2ceb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:55.160469 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c71d73-1a24-43f2-b604-3d5accb2ceb0 None admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:55.160951 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c71d73-1a24-43f2-b604-3d5accb2ceb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:55.161412 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c71d73-1a24-43f2-b604-3d5accb2ceb0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:55.162257 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c71d73-1a24-43f2-b604-3d5accb2ceb0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:55.164697 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c71d73-1a24-43f2-b604-3d5accb2ceb0 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:55.196691 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:55.224172 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b2cf6a-edf4-49e5-a5c4-421c7d5e136d None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:55.262025 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7785c17-45df-4e2d-845d-a4481b4b1386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:55.263034 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7785c17-45df-4e2d-845d-a4481b4b1386 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:55.263708 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7785c17-45df-4e2d-845d-a4481b4b1386 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:55.264062 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 421/825] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:24:54 2022] POST /identity/v3/endpoints => generated 337 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:24:55.264639 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7785c17-45df-4e2d-845d-a4481b4b1386 None admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:55.265638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7785c17-45df-4e2d-845d-a4481b4b1386 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:55.266184 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7785c17-45df-4e2d-845d-a4481b4b1386 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:55.274031 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7785c17-45df-4e2d-845d-a4481b4b1386 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:55.300222 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 405/826] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Aug 24 05:24:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:55.306332 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c71d73-1a24-43f2-b604-3d5accb2ceb0 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:55.416825 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:55.444593 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a41c36-e01f-4ad5-9d08-f725266e656c None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:55.445263 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a41c36-e01f-4ad5-9d08-f725266e656c None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:55.445748 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a41c36-e01f-4ad5-9d08-f725266e656c None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:55.447357 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 406/827] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:55.453852 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ced2b-16ca-440b-8548-cc058c4d43ea None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:55.457005 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ced2b-16ca-440b-8548-cc058c4d43ea None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:55.457005 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ced2b-16ca-440b-8548-cc058c4d43ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:55.477033 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-369ced2b-16ca-440b-8548-cc058c4d43ea None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:55.478258 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-369ced2b-16ca-440b-8548-cc058c4d43ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:55.483067 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c71d73-1a24-43f2-b604-3d5accb2ceb0 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:55.548318 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7355cb20-dda7-4a26-8483-49ebcc376dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:55.548964 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355cb20-dda7-4a26-8483-49ebcc376dd8 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:55.549240 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355cb20-dda7-4a26-8483-49ebcc376dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:55.549510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355cb20-dda7-4a26-8483-49ebcc376dd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:55.603001 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 407/828] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:55 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:24:55.645979 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7355cb20-dda7-4a26-8483-49ebcc376dd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:55.647444 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7355cb20-dda7-4a26-8483-49ebcc376dd8 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:55.734190 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:55.779150 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 422/829] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:24:55 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:24:55.782903 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ced2b-16ca-440b-8548-cc058c4d43ea None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:55.855764 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:55.893524 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ced2b-16ca-440b-8548-cc058c4d43ea None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:55.907875 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7355cb20-dda7-4a26-8483-49ebcc376dd8 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:55.936029 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f372f2b3-4815-43a3-a47b-9d87405b5114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:55.936797 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f372f2b3-4815-43a3-a47b-9d87405b5114 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:55.937091 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f372f2b3-4815-43a3-a47b-9d87405b5114 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:55.937379 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f372f2b3-4815-43a3-a47b-9d87405b5114 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:55.948508 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f372f2b3-4815-43a3-a47b-9d87405b5114 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:55.950078 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f372f2b3-4815-43a3-a47b-9d87405b5114 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:55.967673 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-f372f2b3-4815-43a3-a47b-9d87405b5114 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:55.967673 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:55.967673 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:55.967673 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:55.967673 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:55.967673 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:55.967673 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:55.967673 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:55.967673 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:55.967673 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:55.967673 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:55.967673 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:55.967673 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:55.967673 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:55.967673 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:55.967673 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:55.968743 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:55.968743 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:55.968743 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:55.968743 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:55.968743 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:55.968743 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:55.968743 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:55.968743 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:55.968743 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:55.968743 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:55.968743 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:55.968743 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:55.968743 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:55.968743 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:55.968743 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:55.968743 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:55.968743 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:55.970355 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:55.970355 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:55.970355 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:55.970355 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 408/830] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:55.974649 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f372f2b3-4815-43a3-a47b-9d87405b5114 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:56.037337 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7355cb20-dda7-4a26-8483-49ebcc376dd8 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:56.080783 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f329d5fc-cf99-44c0-b9e2-8fca94ba6fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:56.081427 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329d5fc-cf99-44c0-b9e2-8fca94ba6fcf None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:56.081708 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329d5fc-cf99-44c0-b9e2-8fca94ba6fcf None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:56.081984 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329d5fc-cf99-44c0-b9e2-8fca94ba6fcf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:56.095990 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f329d5fc-cf99-44c0-b9e2-8fca94ba6fcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:24:56.101746 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f372f2b3-4815-43a3-a47b-9d87405b5114 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:56.101883 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-f329d5fc-cf99-44c0-b9e2-8fca94ba6fcf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:24:56.101883 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:56.101883 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:56.101883 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:56.101883 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:56.101883 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:56.101883 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:56.101883 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:56.101883 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:56.101883 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:56.101883 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:56.101883 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:56.101883 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:24:56.101883 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:24:56.101883 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:24:56.101883 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:24:56.102769 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:24:56.102769 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:24:56.102769 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:24:56.102769 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:24:56.102769 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:56.102769 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:56.102769 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:56.102769 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:56.102769 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:56.102769 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:56.102769 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:56.102769 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:56.102769 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:56.102769 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:56.102769 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:56.102769 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:56.104347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:56.104347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:56.104347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:56.104347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:56.104347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:24:56.104347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:24:56.104347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:24:56.104347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:24:56.104347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:24:56.104347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:24:56.104347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:24:56.104347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:56.104347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 423/831] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Aug 24 05:24:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:56.109377 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f329d5fc-cf99-44c0-b9e2-8fca94ba6fcf None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:56.140513 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67ea60-f397-4c41-8e18-35ffc5331cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:56.141164 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67ea60-f397-4c41-8e18-35ffc5331cf3 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:56.141398 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67ea60-f397-4c41-8e18-35ffc5331cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:56.141639 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67ea60-f397-4c41-8e18-35ffc5331cf3 None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:56.142287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac67ea60-f397-4c41-8e18-35ffc5331cf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:56.142618 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac67ea60-f397-4c41-8e18-35ffc5331cf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:56.143966 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac67ea60-f397-4c41-8e18-35ffc5331cf3 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:56.144700 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:56.157216 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 409/832] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:24:56.160791 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:56.162337 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67ea60-f397-4c41-8e18-35ffc5331cf3 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:56.224517 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f329d5fc-cf99-44c0-b9e2-8fca94ba6fcf None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:56.263075 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece609b0-a70d-4162-9363-6696abba04be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:56.264715 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece609b0-a70d-4162-9363-6696abba04be None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:56.265284 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece609b0-a70d-4162-9363-6696abba04be None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:56.265870 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece609b0-a70d-4162-9363-6696abba04be None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:56.266752 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece609b0-a70d-4162-9363-6696abba04be None admin] RBAC: Adding query filter params (name=admin) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:56.267289 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece609b0-a70d-4162-9363-6696abba04be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:56.268900 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece609b0-a70d-4162-9363-6696abba04be None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:56.274567 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67ea60-f397-4c41-8e18-35ffc5331cf3 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:56.278698 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 424/833] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:24:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:56.288525 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece609b0-a70d-4162-9363-6696abba04be None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:56.313160 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700e4398-2099-48ed-950f-43e9f6323e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:56.313801 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e4398-2099-48ed-950f-43e9f6323e14 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:56.314081 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e4398-2099-48ed-950f-43e9f6323e14 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:56.314353 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e4398-2099-48ed-950f-43e9f6323e14 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:56.327394 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700e4398-2099-48ed-950f-43e9f6323e14 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:56.329837 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700e4398-2099-48ed-950f-43e9f6323e14 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:56.344334 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-700e4398-2099-48ed-950f-43e9f6323e14 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:56.344334 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:56.344334 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:56.344334 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:56.344334 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:56.344334 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:56.344334 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:56.344334 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:56.344334 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:56.344334 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:56.344334 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:56.344334 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:56.344334 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:24:56.344334 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:24:56.344334 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:24:56.345277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:24:56.345277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:56.345277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:56.345277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:56.345277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:56.345277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:56.345277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:56.345277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:56.345277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:56.345277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:56.345277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:56.345277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:56.345277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:56.345277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:56.345277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:56.345277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:56.345277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:56.346168 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:24:56.346168 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:24:56.346168 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:24:56.346168 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:24:56.346168 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:24:56.346168 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:24:56.346168 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:56.346168 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:56.346168 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 410/834] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:24:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:56.347250 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a27ab-fd9c-4be2-a3ae-70bce13fb82a None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:56.347250 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a27ab-fd9c-4be2-a3ae-70bce13fb82a None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:56.347535 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a27ab-fd9c-4be2-a3ae-70bce13fb82a None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:56.348928 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 411/835] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:56.351851 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04a27ab-fd9c-4be2-a3ae-70bce13fb82a None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:56.396788 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:56.400426 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:56.410187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece609b0-a70d-4162-9363-6696abba04be None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:56.445293 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f029ff-1cc9-493d-bf78-d5c03aa72778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:56.446293 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f029ff-1cc9-493d-bf78-d5c03aa72778 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:56.446667 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f029ff-1cc9-493d-bf78-d5c03aa72778 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:56.446948 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f029ff-1cc9-493d-bf78-d5c03aa72778 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:56.457672 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f029ff-1cc9-493d-bf78-d5c03aa72778 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:56.459400 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f029ff-1cc9-493d-bf78-d5c03aa72778 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:56.463941 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04a27ab-fd9c-4be2-a3ae-70bce13fb82a None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:56.476299 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-78f029ff-1cc9-493d-bf78-d5c03aa72778 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:56.476299 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:56.476299 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:56.476299 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:56.476299 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:56.476299 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:56.476299 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:56.476299 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:56.476299 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:56.476299 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:56.476299 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:56.476299 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:56.476299 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:56.476299 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:56.476299 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:56.476299 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:56.477507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:56.477507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:56.477507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:56.477507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:56.477507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:56.477507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:56.477507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:56.477507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:56.477507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:56.477507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:56.477507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:56.477507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:56.477507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:56.477507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:56.477507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:56.477507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:56.477507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:56.478621 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:56.478621 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:56.478621 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:56.482304 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 425/836] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:56.484748 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b926ff-51fc-42b0-9df7-d9f0d9084cb8 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:56.485040 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b926ff-51fc-42b0-9df7-d9f0d9084cb8 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:56.485337 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b926ff-51fc-42b0-9df7-d9f0d9084cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:56.503864 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-69b926ff-51fc-42b0-9df7-d9f0d9084cb8 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:56.505109 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-69b926ff-51fc-42b0-9df7-d9f0d9084cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:56.509434 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5659a-b53c-436b-b045-393ec0e64aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:56.510052 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5659a-b53c-436b-b045-393ec0e64aee None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:56.510355 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5659a-b53c-436b-b045-393ec0e64aee None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:56.510601 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5659a-b53c-436b-b045-393ec0e64aee None admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:56.511291 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f5659a-b53c-436b-b045-393ec0e64aee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:56.511596 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f5659a-b53c-436b-b045-393ec0e64aee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:56.513442 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f5659a-b53c-436b-b045-393ec0e64aee None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:56.525568 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 412/837] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Aug 24 05:24:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:56.525568 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5659a-b53c-436b-b045-393ec0e64aee None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:56.592826 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 426/838] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:56 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:24:56.593903 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b926ff-51fc-42b0-9df7-d9f0d9084cb8 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:56.657924 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:56.663908 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5659a-b53c-436b-b045-393ec0e64aee None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:56.676787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:56.710877 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ae5dd6-36fd-4fca-8597-a356add7b396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:56.711472 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ae5dd6-36fd-4fca-8597-a356add7b396 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:56.711759 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ae5dd6-36fd-4fca-8597-a356add7b396 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:56.712036 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ae5dd6-36fd-4fca-8597-a356add7b396 None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:56.712709 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ae5dd6-36fd-4fca-8597-a356add7b396 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:56.713015 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ae5dd6-36fd-4fca-8597-a356add7b396 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:56.714655 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ae5dd6-36fd-4fca-8597-a356add7b396 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:56.718650 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b926ff-51fc-42b0-9df7-d9f0d9084cb8 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:56.727965 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 413/839] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:56.734068 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ae5dd6-36fd-4fca-8597-a356add7b396 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:56.771056 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960887ad-69de-4c0e-8126-d6beb05df1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:56.771749 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960887ad-69de-4c0e-8126-d6beb05df1e9 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:56.772039 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960887ad-69de-4c0e-8126-d6beb05df1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:56.772317 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960887ad-69de-4c0e-8126-d6beb05df1e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:56.773107 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960887ad-69de-4c0e-8126-d6beb05df1e9 None admin] RBAC: Adding query filter params (role.id=e013ad4d436f4cb0a9bf7debb7d13dc7, scope.project.id=59592706ae9442e08353158c625e9581, user.id=fbecffb800584c5786813accd061421b) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:56.773415 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960887ad-69de-4c0e-8126-d6beb05df1e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:56.774870 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960887ad-69de-4c0e-8126-d6beb05df1e9 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:56.781834 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 427/840] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:24:56 2022] GET /identity/v3/role_assignments?user.id=fbecffb800584c5786813accd061421b&scope.project.id=59592706ae9442e08353158c625e9581&role.id=e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:56.860418 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ae5dd6-36fd-4fca-8597-a356add7b396 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:56.881518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960887ad-69de-4c0e-8126-d6beb05df1e9 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:56.906451 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c7cbb-c0c8-457c-b461-3ffc04a85888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:56.908053 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c7cbb-c0c8-457c-b461-3ffc04a85888 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:56.908559 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c7cbb-c0c8-457c-b461-3ffc04a85888 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:56.909133 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c7cbb-c0c8-457c-b461-3ffc04a85888 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:56.921562 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2c7cbb-c0c8-457c-b461-3ffc04a85888 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:56.926209 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2c7cbb-c0c8-457c-b461-3ffc04a85888 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:56.936900 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-aa2c7cbb-c0c8-457c-b461-3ffc04a85888 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 05:24:56.936900 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:56.936900 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:56.936900 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:56.936900 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:56.936900 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:56.936900 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:56.936900 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:56.936900 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:56.936900 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:56.936900 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:56.936900 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:56.936900 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:24:56.936900 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:24:56.936900 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:24:56.936900 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:24:56.937896 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:56.937896 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:56.937896 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:24:56.937896 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:56.937896 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:24:56.937896 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:56.937896 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:56.937896 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:56.937896 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:56.937896 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:56.937896 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:56.937896 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:56.937896 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:56.937896 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:56.937896 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:56.937896 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:56.937896 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:56.937896 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:56.938837 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:56.938837 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:56.938837 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:24:56.938837 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:24:56.938837 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:24:56.938837 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:24:56.938837 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:24:56.938837 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:24:56.938837 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 05:24:56.938837 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:56.938837 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 414/841] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Aug 24 05:24:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:24:56.944336 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c7cbb-c0c8-457c-b461-3ffc04a85888 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:56.968005 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:57.014237 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960887ad-69de-4c0e-8126-d6beb05df1e9 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:57.057489 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c7cbb-c0c8-457c-b461-3ffc04a85888 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:57.061497 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8619cbca-391e-49e5-ba26-e86e7cb17718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:57.062594 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619cbca-391e-49e5-ba26-e86e7cb17718 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:57.063091 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619cbca-391e-49e5-ba26-e86e7cb17718 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:57.063731 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619cbca-391e-49e5-ba26-e86e7cb17718 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:57.064609 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8619cbca-391e-49e5-ba26-e86e7cb17718 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:57.066221 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8619cbca-391e-49e5-ba26-e86e7cb17718 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:57.080345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-8619cbca-391e-49e5-ba26-e86e7cb17718 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 05:24:57.080345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:57.080345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:24:57.080345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:24:57.080345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:24:57.080345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:24:57.080345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:24:57.080345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:24:57.080345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 05:24:57.080345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:57.080345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:24:57.080345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:57.080345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:57.080345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:57.080345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:57.080345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:57.080345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:57.081477 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:57.081477 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:57.081477 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:57.081477 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:57.081477 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:57.081477 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:57.081477 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:24:57.081477 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:24:57.081477 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:24:57.081477 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:24:57.081477 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:57.081477 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:57.081477 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:57.081477 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:57.081477 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:57.081477 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:57.082327 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:57.082327 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:57.082327 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:57.082327 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:57.082327 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:57.082327 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:57.082327 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:57.082327 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:57.082327 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:57.082327 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:57.082327 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:24:57.082327 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:24:57.082327 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 05:24:57.082327 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:57.086031 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 428/842] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Aug 24 05:24:56 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:57.096422 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8619cbca-391e-49e5-ba26-e86e7cb17718 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:57.096783 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:57.102332 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b4f77-0924-4e27-ae72-a139766cb5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:57.102948 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b4f77-0924-4e27-ae72-a139766cb5b5 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:57.109188 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b4f77-0924-4e27-ae72-a139766cb5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:57.109188 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b4f77-0924-4e27-ae72-a139766cb5b5 None admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:57.109188 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0b4f77-0924-4e27-ae72-a139766cb5b5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:57.109188 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0b4f77-0924-4e27-ae72-a139766cb5b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:57.109188 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0b4f77-0924-4e27-ae72-a139766cb5b5 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:57.126283 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 415/843] 10.222.0.15 () {56 vars in 1135 bytes} [Wed Aug 24 05:24:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:57.127547 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0c481-f13c-4bb5-9d0b-b9a13f8bc14e None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:57.127828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0c481-f13c-4bb5-9d0b-b9a13f8bc14e None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:57.128100 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0c481-f13c-4bb5-9d0b-b9a13f8bc14e None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:57.129561 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 416/844] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:57.135046 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df7b01-6214-4394-a3dc-1e91499e53a4 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:57.135337 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df7b01-6214-4394-a3dc-1e91499e53a4 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:57.135620 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df7b01-6214-4394-a3dc-1e91499e53a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:57.154614 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-51df7b01-6214-4394-a3dc-1e91499e53a4 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:57.156344 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-51df7b01-6214-4394-a3dc-1e91499e53a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:57.208572 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8619cbca-391e-49e5-ba26-e86e7cb17718 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:57.235172 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 417/845] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:57 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:24:57.236153 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df7b01-6214-4394-a3dc-1e91499e53a4 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:57.244997 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:57.250909 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d30fc-34db-48a6-b0cb-73a9fc4b0d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:57.257860 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d30fc-34db-48a6-b0cb-73a9fc4b0d50 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:57.257860 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d30fc-34db-48a6-b0cb-73a9fc4b0d50 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:57.257860 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d30fc-34db-48a6-b0cb-73a9fc4b0d50 None admin] PATH_INFO: `/v3/services` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:57.257860 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d30fc-34db-48a6-b0cb-73a9fc4b0d50 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:57.257860 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d30fc-34db-48a6-b0cb-73a9fc4b0d50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:57.257860 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d30fc-34db-48a6-b0cb-73a9fc4b0d50 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:57.262491 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 429/846] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Aug 24 05:24:57 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:57.267739 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d30fc-34db-48a6-b0cb-73a9fc4b0d50 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:57.341100 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:57.367439 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df7b01-6214-4394-a3dc-1e91499e53a4 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:57.388275 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d30fc-34db-48a6-b0cb-73a9fc4b0d50 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:57.407068 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd1c417-8c8f-4487-9d13-b9966178be1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:57.407688 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd1c417-8c8f-4487-9d13-b9966178be1c None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:57.407973 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd1c417-8c8f-4487-9d13-b9966178be1c None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:57.408247 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd1c417-8c8f-4487-9d13-b9966178be1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:57.433369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742109af-3498-4b64-a46c-cdd34d351369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:57.434603 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742109af-3498-4b64-a46c-cdd34d351369 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:57.435138 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742109af-3498-4b64-a46c-cdd34d351369 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:57.436035 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742109af-3498-4b64-a46c-cdd34d351369 None admin] PATH_INFO: `/v3/services` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:57.437448 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742109af-3498-4b64-a46c-cdd34d351369 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:57.438238 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742109af-3498-4b64-a46c-cdd34d351369 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:57.440507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742109af-3498-4b64-a46c-cdd34d351369 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:57.453352 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 430/847] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Aug 24 05:24:57 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:24:57.494483 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd1c417-8c8f-4487-9d13-b9966178be1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:57.495955 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd1c417-8c8f-4487-9d13-b9966178be1c None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:57.505514 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:57.513096 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742109af-3498-4b64-a46c-cdd34d351369 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:57.628578 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 418/848] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:24:57 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:24:57.632023 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742109af-3498-4b64-a46c-cdd34d351369 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:57.633257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd1c417-8c8f-4487-9d13-b9966178be1c None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:57.681222 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:57.681904 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d55b4c-2669-428f-b726-6d0baeb5b8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:57.682556 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d55b4c-2669-428f-b726-6d0baeb5b8f3 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:57.682832 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d55b4c-2669-428f-b726-6d0baeb5b8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:57.683104 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d55b4c-2669-428f-b726-6d0baeb5b8f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:57.698124 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d55b4c-2669-428f-b726-6d0baeb5b8f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:24:57.705076 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-08d55b4c-2669-428f-b726-6d0baeb5b8f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:24:57.705076 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:57.705076 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:57.705076 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:57.705076 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:57.705076 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:57.705076 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:57.705076 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:57.705076 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:57.705076 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:57.705076 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:57.705076 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:57.705076 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:24:57.705076 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:24:57.705076 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:24:57.705076 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:24:57.706010 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:24:57.706010 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:24:57.706010 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:24:57.706010 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:24:57.706010 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:57.706010 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:57.706010 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:57.706010 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:57.706010 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:57.706010 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:57.706010 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:57.706010 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:57.706010 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:57.706010 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:57.706010 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:57.706010 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:57.706856 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:57.706856 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:57.706856 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:57.706856 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:57.706856 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:24:57.706856 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:24:57.706856 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:24:57.706856 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:24:57.706856 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:24:57.706856 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:24:57.706856 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:24:57.706856 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:57.708548 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 431/849] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Aug 24 05:24:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:57.712217 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d55b4c-2669-428f-b726-6d0baeb5b8f3 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:57.755094 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd1c417-8c8f-4487-9d13-b9966178be1c None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:57.764537 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:57.793398 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172dcad1-3661-4a65-b53e-2cb7407ccdd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:57.794484 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172dcad1-3661-4a65-b53e-2cb7407ccdd3 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:57.795174 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172dcad1-3661-4a65-b53e-2cb7407ccdd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:57.795664 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172dcad1-3661-4a65-b53e-2cb7407ccdd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:57.807041 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172dcad1-3661-4a65-b53e-2cb7407ccdd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:57.808875 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172dcad1-3661-4a65-b53e-2cb7407ccdd3 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:57.825679 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-172dcad1-3661-4a65-b53e-2cb7407ccdd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:57.825679 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:57.825679 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:57.825679 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:57.825679 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:57.825679 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:57.825679 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:57.825679 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:57.825679 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:57.825679 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:57.825679 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:57.825679 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:57.825679 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:57.825679 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:57.825679 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:57.825679 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:57.827465 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:57.827465 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:57.827465 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:57.827465 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:57.827465 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:57.827465 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:57.827465 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:57.827465 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:57.827465 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:57.827465 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:57.827465 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:57.827465 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:57.827465 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:57.827465 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:57.827465 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:57.827465 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:57.827465 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:57.828355 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:57.828355 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:57.828355 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:57.830757 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 419/850] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:57.836006 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172dcad1-3661-4a65-b53e-2cb7407ccdd3 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:57.844668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d55b4c-2669-428f-b726-6d0baeb5b8f3 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:57.883845 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00c7961-8a48-45bc-b02a-1d24bc9c91f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:57.885107 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c7961-8a48-45bc-b02a-1d24bc9c91f4 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:57.885394 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c7961-8a48-45bc-b02a-1d24bc9c91f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:57.885673 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c7961-8a48-45bc-b02a-1d24bc9c91f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:57.886342 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00c7961-8a48-45bc-b02a-1d24bc9c91f4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:57.886651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00c7961-8a48-45bc-b02a-1d24bc9c91f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:57.888023 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00c7961-8a48-45bc-b02a-1d24bc9c91f4 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:57.901319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 432/851] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Aug 24 05:24:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:57.902263 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00c7961-8a48-45bc-b02a-1d24bc9c91f4 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:57.951771 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172dcad1-3661-4a65-b53e-2cb7407ccdd3 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:57.960243 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:57.996218 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f53bef-1d2c-4cba-93a6-a42359d15a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:57.996901 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f53bef-1d2c-4cba-93a6-a42359d15a2e None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:57.997192 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f53bef-1d2c-4cba-93a6-a42359d15a2e None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:57.997443 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f53bef-1d2c-4cba-93a6-a42359d15a2e None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:57.998659 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f53bef-1d2c-4cba-93a6-a42359d15a2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:57.999090 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f53bef-1d2c-4cba-93a6-a42359d15a2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:58.000623 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f53bef-1d2c-4cba-93a6-a42359d15a2e None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:58.018399 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 420/852] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:58.023389 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f53bef-1d2c-4cba-93a6-a42359d15a2e None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:58.042854 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00c7961-8a48-45bc-b02a-1d24bc9c91f4 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:58.044101 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:58.093532 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01909435-3317-439a-9116-05cd96cb08f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:58.094160 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01909435-3317-439a-9116-05cd96cb08f7 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:58.094443 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01909435-3317-439a-9116-05cd96cb08f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:58.094717 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01909435-3317-439a-9116-05cd96cb08f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:58.111157 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01909435-3317-439a-9116-05cd96cb08f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:58.113740 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01909435-3317-439a-9116-05cd96cb08f7 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:58.135048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-01909435-3317-439a-9116-05cd96cb08f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:58.135048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:58.135048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:58.135048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:58.135048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:58.135048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:58.135048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:58.135048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:58.135048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:58.135048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:58.135048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:58.135048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:58.135048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:58.135048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:58.135048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:58.135048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:58.136043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:58.136043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:58.136043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:58.136043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:58.136043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:58.136043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:58.136043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:58.136043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:58.136043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:58.136043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:58.136043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:58.136043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:58.136043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:58.136043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:58.136043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:58.136043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:58.136043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:58.136961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:58.136961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:58.136961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:58.137131 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f53bef-1d2c-4cba-93a6-a42359d15a2e None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:58.140702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 433/853] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:58.149375 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01909435-3317-439a-9116-05cd96cb08f7 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:58.184856 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6be5339-ab4d-466b-9e06-b9fc988002e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:58.185914 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be5339-ab4d-466b-9e06-b9fc988002e2 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:58.186400 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be5339-ab4d-466b-9e06-b9fc988002e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:58.186892 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be5339-ab4d-466b-9e06-b9fc988002e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:58.197775 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6be5339-ab4d-466b-9e06-b9fc988002e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:58.199443 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6be5339-ab4d-466b-9e06-b9fc988002e2 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:58.215417 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-d6be5339-ab4d-466b-9e06-b9fc988002e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:58.215417 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:58.215417 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:58.215417 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:58.215417 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:58.215417 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:58.215417 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:58.215417 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:58.215417 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:58.215417 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:58.215417 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:58.215417 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:58.215417 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:24:58.215417 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:24:58.215417 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:24:58.216333 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:24:58.216333 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:58.216333 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:58.216333 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:58.216333 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:58.216333 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:58.216333 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:58.216333 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:58.216333 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:58.216333 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:58.216333 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:58.216333 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:58.216333 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:58.216333 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:58.216333 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:58.216333 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:58.216333 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:58.217680 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:24:58.217680 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:24:58.217680 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:24:58.217680 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:24:58.217680 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:24:58.217680 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:24:58.217680 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:58.217680 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:58.222219 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 421/854] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:24:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:58.224348 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6be5339-ab4d-466b-9e06-b9fc988002e2 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:58.284444 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01909435-3317-439a-9116-05cd96cb08f7 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:58.287797 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:58.315501 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:58.333859 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ad1e4-2612-498c-8d09-cf55c619326f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:58.334773 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ad1e4-2612-498c-8d09-cf55c619326f None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:58.335176 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ad1e4-2612-498c-8d09-cf55c619326f None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:58.335464 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ad1e4-2612-498c-8d09-cf55c619326f None admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:58.336126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772ad1e4-2612-498c-8d09-cf55c619326f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:58.336461 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772ad1e4-2612-498c-8d09-cf55c619326f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:58.338657 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772ad1e4-2612-498c-8d09-cf55c619326f None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:58.344594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6be5339-ab4d-466b-9e06-b9fc988002e2 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:58.358678 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 434/855] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:58.360298 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ad1e4-2612-498c-8d09-cf55c619326f None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:58.391947 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd229194-a9d7-49f6-a9a1-de1572359311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:58.392501 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd229194-a9d7-49f6-a9a1-de1572359311 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:58.392789 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd229194-a9d7-49f6-a9a1-de1572359311 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:58.393063 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd229194-a9d7-49f6-a9a1-de1572359311 None admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:58.393760 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd229194-a9d7-49f6-a9a1-de1572359311 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:58.394069 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd229194-a9d7-49f6-a9a1-de1572359311 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:58.395526 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd229194-a9d7-49f6-a9a1-de1572359311 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:58.414654 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 422/856] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Aug 24 05:24:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:58.415570 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd229194-a9d7-49f6-a9a1-de1572359311 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:58.498887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ad1e4-2612-498c-8d09-cf55c619326f None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:58.535106 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd229194-a9d7-49f6-a9a1-de1572359311 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:58.545451 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4bb1f6-98a6-4820-aca3-1d1f8c59f806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:58.546542 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4bb1f6-98a6-4820-aca3-1d1f8c59f806 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:58.547539 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4bb1f6-98a6-4820-aca3-1d1f8c59f806 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:58.548039 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4bb1f6-98a6-4820-aca3-1d1f8c59f806 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:58.560191 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4bb1f6-98a6-4820-aca3-1d1f8c59f806 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:58.563154 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4bb1f6-98a6-4820-aca3-1d1f8c59f806 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:58.579908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-2e4bb1f6-98a6-4820-aca3-1d1f8c59f806 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 05:24:58.579908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:58.579908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:58.579908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:58.579908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:58.579908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:58.579908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:58.579908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:58.579908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:58.579908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:58.579908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:58.579908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:58.579908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:24:58.579908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:24:58.579908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:24:58.579908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:24:58.580955 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:58.580955 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:58.580955 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:24:58.580955 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:58.580955 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:24:58.580955 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:24:58.580955 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:58.580955 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:58.580955 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:58.580955 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:58.580955 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:58.580955 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:58.580955 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:58.580955 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:58.580955 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:58.580955 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:58.580955 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:58.580955 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:58.581895 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:58.581895 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:58.581895 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:24:58.581895 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:24:58.581895 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:24:58.581895 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:24:58.581895 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:24:58.581895 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:24:58.581895 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 05:24:58.581895 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:58.585787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 435/857] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Aug 24 05:24:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:24:58.586689 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4bb1f6-98a6-4820-aca3-1d1f8c59f806 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:58.590482 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f503b79f-9ee6-4b6a-826c-77abf22d96e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:58.591414 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f503b79f-9ee6-4b6a-826c-77abf22d96e1 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:58.592048 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f503b79f-9ee6-4b6a-826c-77abf22d96e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:58.592656 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f503b79f-9ee6-4b6a-826c-77abf22d96e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:58.593718 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f503b79f-9ee6-4b6a-826c-77abf22d96e1 None admin] RBAC: Adding query filter params (role.id=e013ad4d436f4cb0a9bf7debb7d13dc7, scope.project.id=59592706ae9442e08353158c625e9581, user.id=bb2b6cae4a004539a95fc7dd558db255) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:58.594248 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f503b79f-9ee6-4b6a-826c-77abf22d96e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:58.595979 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f503b79f-9ee6-4b6a-826c-77abf22d96e1 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:58.604274 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 423/858] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:24:58 2022] GET /identity/v3/role_assignments?user.id=bb2b6cae4a004539a95fc7dd558db255&scope.project.id=59592706ae9442e08353158c625e9581&role.id=e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:58.626165 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:58.713495 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4bb1f6-98a6-4820-aca3-1d1f8c59f806 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:58.753136 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a97d0a2-cd69-4c6d-8dee-fce48514ace2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:58.754155 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a97d0a2-cd69-4c6d-8dee-fce48514ace2 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:58.754664 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a97d0a2-cd69-4c6d-8dee-fce48514ace2 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:58.755143 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a97d0a2-cd69-4c6d-8dee-fce48514ace2 None admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:58.756126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a97d0a2-cd69-4c6d-8dee-fce48514ace2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:58.756672 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a97d0a2-cd69-4c6d-8dee-fce48514ace2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:58.758626 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a97d0a2-cd69-4c6d-8dee-fce48514ace2 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:58.780390 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 436/859] 10.222.0.15 () {56 vars in 1149 bytes} [Wed Aug 24 05:24:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:58.789586 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f503b79f-9ee6-4b6a-826c-77abf22d96e1 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:58.792789 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:58.840349 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff345967-9886-4903-bca3-178baf93b9c0 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:58.840898 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff345967-9886-4903-bca3-178baf93b9c0 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:58.841268 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff345967-9886-4903-bca3-178baf93b9c0 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:58.843228 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 437/860] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:58.848798 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a737b888-7804-4ac3-a004-6c16861b5b57 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:58.849186 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a737b888-7804-4ac3-a004-6c16861b5b57 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:58.849536 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a737b888-7804-4ac3-a004-6c16861b5b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:58.871031 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-a737b888-7804-4ac3-a004-6c16861b5b57 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:58.872249 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-a737b888-7804-4ac3-a004-6c16861b5b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:58.919436 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f503b79f-9ee6-4b6a-826c-77abf22d96e1 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:58.956582 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 438/861] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:58 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:24:58.964961 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccea1723-453a-4f17-9172-7309062e9d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:58.965800 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccea1723-453a-4f17-9172-7309062e9d1d None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:58.966277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccea1723-453a-4f17-9172-7309062e9d1d None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:58.966767 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccea1723-453a-4f17-9172-7309062e9d1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:59.039962 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccea1723-453a-4f17-9172-7309062e9d1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:59.041763 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccea1723-453a-4f17-9172-7309062e9d1d None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:59.043743 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:59.153826 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 424/862] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:24:58 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 05:24:59.161851 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a737b888-7804-4ac3-a004-6c16861b5b57 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:59.179771 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:59.245025 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccea1723-453a-4f17-9172-7309062e9d1d None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:59.282445 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a737b888-7804-4ac3-a004-6c16861b5b57 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:59.325988 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21102f94-d7bd-4130-8f1d-4c211d3d5b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:59.327172 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21102f94-d7bd-4130-8f1d-4c211d3d5b89 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:59.327662 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21102f94-d7bd-4130-8f1d-4c211d3d5b89 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:59.328139 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21102f94-d7bd-4130-8f1d-4c211d3d5b89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:59.339146 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21102f94-d7bd-4130-8f1d-4c211d3d5b89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:59.340925 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21102f94-d7bd-4130-8f1d-4c211d3d5b89 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:59.357652 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-21102f94-d7bd-4130-8f1d-4c211d3d5b89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:59.357652 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:59.357652 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:59.357652 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:59.357652 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:59.357652 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:59.357652 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:59.357652 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:59.357652 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:59.357652 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:59.357652 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:59.357652 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:59.357652 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:24:59.357652 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:24:59.357652 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:24:59.357652 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:24:59.358764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:59.358764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:59.358764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:59.358764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:59.358764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:59.358764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:59.358764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:59.358764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:59.358764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:59.358764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:59.358764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:59.358764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:59.358764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:59.358764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:59.358764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:59.358764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:59.358764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:24:59.359657 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:24:59.359657 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:24:59.359657 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:59.365940 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccea1723-453a-4f17-9172-7309062e9d1d None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:59.365940 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:59.366625 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 439/863] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:24:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:59.368135 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21102f94-d7bd-4130-8f1d-4c211d3d5b89 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:59.410314 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97524c34-96ca-480e-b0ef-16d212091c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:59.411353 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97524c34-96ca-480e-b0ef-16d212091c82 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:59.411928 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97524c34-96ca-480e-b0ef-16d212091c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:59.412427 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97524c34-96ca-480e-b0ef-16d212091c82 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:59.413224 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97524c34-96ca-480e-b0ef-16d212091c82 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:59.414854 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97524c34-96ca-480e-b0ef-16d212091c82 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:59.424469 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-97524c34-96ca-480e-b0ef-16d212091c82 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 05:24:59.424469 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:59.424469 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:24:59.424469 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:24:59.424469 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:24:59.424469 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:24:59.424469 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:24:59.424469 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:24:59.424469 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 05:24:59.424469 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:59.424469 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:24:59.424469 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:59.424469 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:59.424469 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:59.424469 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:59.424469 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:59.425441 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:59.425441 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:59.425441 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:59.425441 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:59.425441 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:59.425441 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:59.425441 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:59.425441 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:24:59.425441 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:24:59.425441 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:24:59.425441 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:24:59.425441 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:59.425441 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:59.425441 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:59.425441 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:59.425441 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:59.425441 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:59.426354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:59.426354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:59.426354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:59.426354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:59.426354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:59.426354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:59.426354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:59.426354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:59.426354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:59.426354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:59.426354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:24:59.426354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:24:59.426354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 05:24:59.426354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:24:59.433725 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 425/864] 10.222.0.15 () {56 vars in 1132 bytes} [Wed Aug 24 05:24:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:59.435745 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97524c34-96ca-480e-b0ef-16d212091c82 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:59.439198 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:59.483528 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21102f94-d7bd-4130-8f1d-4c211d3d5b89 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:59.519867 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d4cba-7700-4708-b5f7-871bde119b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:59.520510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d4cba-7700-4708-b5f7-871bde119b73 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:59.520798 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d4cba-7700-4708-b5f7-871bde119b73 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:59.521070 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d4cba-7700-4708-b5f7-871bde119b73 None admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:59.521707 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9d4cba-7700-4708-b5f7-871bde119b73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:59.522008 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9d4cba-7700-4708-b5f7-871bde119b73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:59.523370 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9d4cba-7700-4708-b5f7-871bde119b73 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:59.540140 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 440/865] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:24:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:24:59.540140 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba1392-390d-4531-80bf-5a912643939d None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:59.540140 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba1392-390d-4531-80bf-5a912643939d None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:59.540140 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba1392-390d-4531-80bf-5a912643939d None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:59.540140 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 441/866] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:24:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:24:59.541491 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba1392-390d-4531-80bf-5a912643939d None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:59.556175 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97524c34-96ca-480e-b0ef-16d212091c82 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:59.591522 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45f9aa-c66b-4b1b-8d91-0122b7a2ac71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:59.592099 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45f9aa-c66b-4b1b-8d91-0122b7a2ac71 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:59.592378 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45f9aa-c66b-4b1b-8d91-0122b7a2ac71 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:59.592680 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45f9aa-c66b-4b1b-8d91-0122b7a2ac71 None admin] PATH_INFO: `/v3/services` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:59.593302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f45f9aa-c66b-4b1b-8d91-0122b7a2ac71 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:59.593602 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f45f9aa-c66b-4b1b-8d91-0122b7a2ac71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:59.595013 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f45f9aa-c66b-4b1b-8d91-0122b7a2ac71 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:59.600176 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 426/867] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Aug 24 05:24:59 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:24:59.601242 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8c84d-88df-4fad-8c40-b3c5fc5e1ae3 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:59.601525 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8c84d-88df-4fad-8c40-b3c5fc5e1ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:59.601802 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8c84d-88df-4fad-8c40-b3c5fc5e1ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:59.620286 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-bea8c84d-88df-4fad-8c40-b3c5fc5e1ae3 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:24:59.621521 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-bea8c84d-88df-4fad-8c40-b3c5fc5e1ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:24:59.646288 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:59.672708 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba1392-390d-4531-80bf-5a912643939d None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:59.701304 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:24:59.701962 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 427/868] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:24:59 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:24:59.703240 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea8c84d-88df-4fad-8c40-b3c5fc5e1ae3 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:59.713356 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea108771-9d5b-41c9-8869-92511896a04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:59.714531 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea108771-9d5b-41c9-8869-92511896a04e None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:59.714813 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea108771-9d5b-41c9-8869-92511896a04e None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:59.715142 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea108771-9d5b-41c9-8869-92511896a04e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:59.725105 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea108771-9d5b-41c9-8869-92511896a04e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:59.726638 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea108771-9d5b-41c9-8869-92511896a04e None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:59.748072 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-ea108771-9d5b-41c9-8869-92511896a04e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:59.748072 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:24:59.748072 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:24:59.748072 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:24:59.748072 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:24:59.748072 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:24:59.748072 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:24:59.748072 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:24:59.748072 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:24:59.748072 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:24:59.748072 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:24:59.748072 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:24:59.748072 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:24:59.748072 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:24:59.748072 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:24:59.749211 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:24:59.749211 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:24:59.749211 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:24:59.749211 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:24:59.749211 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:24:59.749211 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:24:59.749211 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:24:59.749211 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:24:59.749211 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:24:59.749211 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:24:59.749211 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:24:59.749211 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:24:59.749211 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:24:59.749211 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:24:59.749211 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:24:59.749211 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:24:59.749211 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:24:59.750749 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:24:59.750749 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:24:59.750749 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:24:59.750749 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:24:59.750749 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:24:59.750749 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:24:59.750749 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:24:59.750749 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:24:59.750749 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 442/869] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:24:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:24:59.754903 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea108771-9d5b-41c9-8869-92511896a04e None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:24:59.830168 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea8c84d-88df-4fad-8c40-b3c5fc5e1ae3 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:59.871674 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35124e6d-042f-48b8-aed5-d2875a9f62ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:59.872665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35124e6d-042f-48b8-aed5-d2875a9f62ff None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:59.873815 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35124e6d-042f-48b8-aed5-d2875a9f62ff None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:59.874406 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35124e6d-042f-48b8-aed5-d2875a9f62ff None admin] PATH_INFO: `/v3/services` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:59.875717 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35124e6d-042f-48b8-aed5-d2875a9f62ff None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:59.876232 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35124e6d-042f-48b8-aed5-d2875a9f62ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:59.878066 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35124e6d-042f-48b8-aed5-d2875a9f62ff None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:59.884942 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 428/870] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Aug 24 05:24:59 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:24:59.902174 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea108771-9d5b-41c9-8869-92511896a04e None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:24:59.946406 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc4351-8549-42cf-9c1a-ce318e04c008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:24:59.947035 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc4351-8549-42cf-9c1a-ce318e04c008 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:24:59.947388 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc4351-8549-42cf-9c1a-ce318e04c008 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:24:59.947663 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc4351-8549-42cf-9c1a-ce318e04c008 None admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:24:59.948443 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdc4351-8549-42cf-9c1a-ce318e04c008 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:24:59.948765 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdc4351-8549-42cf-9c1a-ce318e04c008 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:24:59.950250 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdc4351-8549-42cf-9c1a-ce318e04c008 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:24:59.965751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 443/871] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Aug 24 05:24:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:24:59.971871 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35124e6d-042f-48b8-aed5-d2875a9f62ff None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:00.004176 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:00.025174 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc4351-8549-42cf-9c1a-ce318e04c008 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:00.089856 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:00.097070 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35124e6d-042f-48b8-aed5-d2875a9f62ff None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:00.146692 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88a85f5-8d7d-436e-8b9c-953c896cc188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:00.147366 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a85f5-8d7d-436e-8b9c-953c896cc188 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:00.147988 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a85f5-8d7d-436e-8b9c-953c896cc188 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:00.148271 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a85f5-8d7d-436e-8b9c-953c896cc188 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:00.149326 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88a85f5-8d7d-436e-8b9c-953c896cc188 None admin] RBAC: Adding query filter params (role.id=cb40cd8499f7418ca0da5ea0f1b87ff9, scope.project.id=59592706ae9442e08353158c625e9581, user.id=604380c191c8461bbb937b8d951238bc) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:00.149642 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88a85f5-8d7d-436e-8b9c-953c896cc188 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:00.151569 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88a85f5-8d7d-436e-8b9c-953c896cc188 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:00.154884 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc4351-8549-42cf-9c1a-ce318e04c008 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:00.164389 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 429/872] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:24:59 2022] GET /identity/v3/role_assignments?user.id=604380c191c8461bbb937b8d951238bc&scope.project.id=59592706ae9442e08353158c625e9581&role.id=cb40cd8499f7418ca0da5ea0f1b87ff9 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:00.206936 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2349e1b7-4cfc-4a15-b3cd-391d1252570e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:00.208222 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349e1b7-4cfc-4a15-b3cd-391d1252570e None admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:00.208712 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349e1b7-4cfc-4a15-b3cd-391d1252570e None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:00.209170 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349e1b7-4cfc-4a15-b3cd-391d1252570e None admin] PATH_INFO: `/v3/services` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:00.210018 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2349e1b7-4cfc-4a15-b3cd-391d1252570e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:00.211619 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2349e1b7-4cfc-4a15-b3cd-391d1252570e None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:00.287483 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 444/873] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Aug 24 05:25:00 2022] POST /identity/v3/services => generated 251 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:25:00.733511 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0178e73-424a-4f58-871a-0c22b04dfd3e None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:00.734124 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0178e73-424a-4f58-871a-0c22b04dfd3e None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:00.734124 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0178e73-424a-4f58-871a-0c22b04dfd3e None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:00.735622 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 430/874] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:25:00.741321 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12508518-bff8-41d2-b9b6-6750eeaa5827 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:00.741614 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12508518-bff8-41d2-b9b6-6750eeaa5827 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:00.741898 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12508518-bff8-41d2-b9b6-6750eeaa5827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:00.761191 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-12508518-bff8-41d2-b9b6-6750eeaa5827 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:25:00.762394 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-12508518-bff8-41d2-b9b6-6750eeaa5827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:25:00.840497 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 445/875] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:25:00 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:25:01.117575 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0178e73-424a-4f58-871a-0c22b04dfd3e None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:01.213450 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:01.230277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0178e73-424a-4f58-871a-0c22b04dfd3e None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:01.270580 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e0701-ebb6-4ea6-a590-8c80a0c447a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:01.271216 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e0701-ebb6-4ea6-a590-8c80a0c447a6 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:01.271499 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e0701-ebb6-4ea6-a590-8c80a0c447a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:01.271771 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e0701-ebb6-4ea6-a590-8c80a0c447a6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:01.272386 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79e0701-ebb6-4ea6-a590-8c80a0c447a6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:01.273747 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79e0701-ebb6-4ea6-a590-8c80a0c447a6 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:01.283277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-e79e0701-ebb6-4ea6-a590-8c80a0c447a6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:25:01.283277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:01.283277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:25:01.283277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:25:01.283277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:25:01.283277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:25:01.283277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:25:01.283277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:25:01.283277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:25:01.283277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:25:01.283277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:25:01.283277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:25:01.283277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:01.283277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:01.283277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:01.283277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:01.283277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:01.284317 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:01.284317 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:01.284317 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:01.284317 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:01.284317 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:01.284317 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:01.284317 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:25:01.284317 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:25:01.284317 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:25:01.284317 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:25:01.284317 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:01.284317 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:01.284317 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:01.284317 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:01.284317 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:01.284317 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:01.285352 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:01.285352 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:01.285352 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:01.285352 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:01.285352 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:01.285352 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:01.285352 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:01.285352 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:01.285352 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:01.285352 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:01.285352 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:25:01.285352 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:25:01.285352 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:25:01.285352 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:25:01.288764 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 431/876] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Aug 24 05:25:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:25:01.290565 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12508518-bff8-41d2-b9b6-6750eeaa5827 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:01.322611 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:01.402725 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12508518-bff8-41d2-b9b6-6750eeaa5827 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:01.445142 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e224ee-7633-4637-b94f-836bc770aaad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:01.446127 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e224ee-7633-4637-b94f-836bc770aaad None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:01.446620 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e224ee-7633-4637-b94f-836bc770aaad None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:01.447105 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e224ee-7633-4637-b94f-836bc770aaad None admin] PATH_INFO: `/v3/services` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:01.447976 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e224ee-7633-4637-b94f-836bc770aaad None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:01.448525 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e224ee-7633-4637-b94f-836bc770aaad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:01.450163 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e224ee-7633-4637-b94f-836bc770aaad None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:01.456243 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 446/877] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Aug 24 05:25:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:01.463569 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e0701-ebb6-4ea6-a590-8c80a0c447a6 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:01.570780 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e0701-ebb6-4ea6-a590-8c80a0c447a6 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:01.608631 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a2582a-45e5-4718-9ca8-17466c0b2222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:01.609587 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2582a-45e5-4718-9ca8-17466c0b2222 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:01.610074 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2582a-45e5-4718-9ca8-17466c0b2222 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:01.610567 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2582a-45e5-4718-9ca8-17466c0b2222 None admin] PATH_INFO: `/v3/services` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:01.611412 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a2582a-45e5-4718-9ca8-17466c0b2222 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:01.611934 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a2582a-45e5-4718-9ca8-17466c0b2222 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:01.613550 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a2582a-45e5-4718-9ca8-17466c0b2222 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:01.618934 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 432/878] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Aug 24 05:25:01 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:01.990911 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d310f4-dadd-4736-922a-89dacc8ff029 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:01.991292 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d310f4-dadd-4736-922a-89dacc8ff029 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:01.991499 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d310f4-dadd-4736-922a-89dacc8ff029 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:01.993134 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 447/879] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:25:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:25:01.998939 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd816c3-87e5-4a0f-8549-8c316919069a None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:01.999419 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd816c3-87e5-4a0f-8549-8c316919069a None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:01.999497 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd816c3-87e5-4a0f-8549-8c316919069a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:02.019221 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-0dd816c3-87e5-4a0f-8549-8c316919069a None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:25:02.020467 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-0dd816c3-87e5-4a0f-8549-8c316919069a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:25:02.032557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:02.091759 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 433/880] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:25:01 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:25:02.228501 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9ed23-14b8-4c01-b35b-9a05e695a299 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:02.228859 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9ed23-14b8-4c01-b35b-9a05e695a299 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:02.229060 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9ed23-14b8-4c01-b35b-9a05e695a299 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:02.230649 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 448/881] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:25:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:25:02.236191 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1ac6b-4023-4d86-9480-d464f7468ed9 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:02.236508 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1ac6b-4023-4d86-9480-d464f7468ed9 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:02.236788 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1ac6b-4023-4d86-9480-d464f7468ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:02.255717 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-4cd1ac6b-4023-4d86-9480-d464f7468ed9 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:25:02.256878 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-4cd1ac6b-4023-4d86-9480-d464f7468ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:25:02.328481 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 434/882] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:25:02 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:25:02.359596 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e41d95-b0c3-4379-a781-b7f0398ee803 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:02.360158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e41d95-b0c3-4379-a781-b7f0398ee803 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:02.360238 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e41d95-b0c3-4379-a781-b7f0398ee803 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:02.361836 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 449/883] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:25:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:25:02.363987 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1ac6b-4023-4d86-9480-d464f7468ed9 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:02.367387 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808b3bb-b588-4aee-9c9b-7c41aa38228b None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:02.367675 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808b3bb-b588-4aee-9c9b-7c41aa38228b None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:02.367957 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808b3bb-b588-4aee-9c9b-7c41aa38228b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:02.386373 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-c808b3bb-b588-4aee-9c9b-7c41aa38228b None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:25:02.387695 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-c808b3bb-b588-4aee-9c9b-7c41aa38228b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:25:02.460521 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 450/884] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:25:02 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:25:02.461192 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:02.484312 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1ac6b-4023-4d86-9480-d464f7468ed9 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:02.519451 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535d7d2a-4288-49a4-97a0-45eedd48f161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:02.520027 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d7d2a-4288-49a4-97a0-45eedd48f161 None admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:02.520357 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d7d2a-4288-49a4-97a0-45eedd48f161 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:02.520663 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d7d2a-4288-49a4-97a0-45eedd48f161 None admin] PATH_INFO: `/v3/services` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:02.521327 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535d7d2a-4288-49a4-97a0-45eedd48f161 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:02.522700 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535d7d2a-4288-49a4-97a0-45eedd48f161 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:02.607651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c808b3bb-b588-4aee-9c9b-7c41aa38228b None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:02.637351 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:02.645754 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 435/885] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Aug 24 05:25:02 2022] POST /identity/v3/services => generated 269 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:25:02.728166 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c808b3bb-b588-4aee-9c9b-7c41aa38228b None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:02.766836 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535d7d2a-4288-49a4-97a0-45eedd48f161 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:02.774562 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdca159-ee41-4316-972a-1f4393286cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:02.775131 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdca159-ee41-4316-972a-1f4393286cda None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:02.775412 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdca159-ee41-4316-972a-1f4393286cda None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:02.775685 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdca159-ee41-4316-972a-1f4393286cda None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:02.790305 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdca159-ee41-4316-972a-1f4393286cda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:25:02.795835 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-fcdca159-ee41-4316-972a-1f4393286cda None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 05:25:02.795835 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:02.795835 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:02.795835 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:02.795835 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:02.795835 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:02.795835 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:02.795835 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:02.795835 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:02.795835 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:02.795835 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:02.795835 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:02.795835 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:25:02.795835 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:25:02.795835 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:25:02.795835 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:25:02.797831 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:25:02.797831 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:25:02.797831 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:25:02.797831 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:25:02.797831 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:02.797831 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:02.797831 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:02.797831 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:02.797831 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:02.797831 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:02.797831 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:02.797831 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:02.797831 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:02.797831 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:02.797831 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:02.797831 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:02.798793 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:02.798793 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:02.798793 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:02.798793 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:02.798793 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:25:02.798793 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:25:02.798793 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:25:02.798793 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:25:02.798793 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:25:02.798793 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:25:02.798793 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 05:25:02.798793 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:02.798793 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 451/886] 10.222.0.15 () {56 vars in 1120 bytes} [Wed Aug 24 05:25:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:25:02.803672 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdca159-ee41-4316-972a-1f4393286cda None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:02.900193 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:02.910647 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535d7d2a-4288-49a4-97a0-45eedd48f161 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:02.925092 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdca159-ee41-4316-972a-1f4393286cda None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:02.954314 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2f5214-be37-489a-bdaa-034de9978a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:02.955305 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f5214-be37-489a-bdaa-034de9978a31 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:02.955646 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f5214-be37-489a-bdaa-034de9978a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:02.955981 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f5214-be37-489a-bdaa-034de9978a31 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:02.956773 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2f5214-be37-489a-bdaa-034de9978a31 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:02.958386 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2f5214-be37-489a-bdaa-034de9978a31 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:02.961797 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:02.970010 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-1a2f5214-be37-489a-bdaa-034de9978a31 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 05:25:02.970010 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:02.970010 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:25:02.970010 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:25:02.970010 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:25:02.970010 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:25:02.970010 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:25:02.970010 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:25:02.970010 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 05:25:02.970010 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:25:02.970010 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:25:02.970010 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:25:02.970010 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:02.970010 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:02.970010 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:02.970010 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:02.970010 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:02.978730 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:02.978730 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:02.978730 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:02.978730 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:02.978730 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:02.978730 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:02.978730 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:25:02.978730 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:25:02.978730 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:25:02.978730 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:25:02.978730 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:02.978730 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:02.978730 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:02.978730 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:02.978730 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:02.978730 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:02.980272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:02.980272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:02.980272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:02.980272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:02.980272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:02.980272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:02.980272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:02.980272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:02.980272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:02.980272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:02.980272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:25:02.980272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:25:02.980272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 05:25:02.980272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:25:02.980272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 436/887] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Aug 24 05:25:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:25:02.981725 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea09a8a9-1cde-4067-a514-0f12fcea9374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:02.981725 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09a8a9-1cde-4067-a514-0f12fcea9374 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:02.981725 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09a8a9-1cde-4067-a514-0f12fcea9374 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:02.981725 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09a8a9-1cde-4067-a514-0f12fcea9374 None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:02.982779 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea09a8a9-1cde-4067-a514-0f12fcea9374 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:02.982779 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea09a8a9-1cde-4067-a514-0f12fcea9374 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:02.982779 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea09a8a9-1cde-4067-a514-0f12fcea9374 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:02.985539 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2f5214-be37-489a-bdaa-034de9978a31 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:02.996029 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 452/888] 10.222.0.15 () {56 vars in 1115 bytes} [Wed Aug 24 05:25:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:03.002005 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea09a8a9-1cde-4067-a514-0f12fcea9374 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:03.113887 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2f5214-be37-489a-bdaa-034de9978a31 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:03.115092 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea09a8a9-1cde-4067-a514-0f12fcea9374 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:03.154923 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422292f4-7a06-48d0-898a-0cbc4212060e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:03.155565 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422292f4-7a06-48d0-898a-0cbc4212060e None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:03.155845 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422292f4-7a06-48d0-898a-0cbc4212060e None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:03.156378 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422292f4-7a06-48d0-898a-0cbc4212060e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:03.164062 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece6e23-f622-4333-99d7-27551e39b931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:03.165050 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece6e23-f622-4333-99d7-27551e39b931 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:03.165551 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece6e23-f622-4333-99d7-27551e39b931 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:03.166143 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece6e23-f622-4333-99d7-27551e39b931 None admin] PATH_INFO: `/v3/services` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:03.167037 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aece6e23-f622-4333-99d7-27551e39b931 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:03.167537 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aece6e23-f622-4333-99d7-27551e39b931 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:03.168267 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422292f4-7a06-48d0-898a-0cbc4212060e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:03.169396 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aece6e23-f622-4333-99d7-27551e39b931 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:03.169760 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422292f4-7a06-48d0-898a-0cbc4212060e None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:03.171757 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:03.176916 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 437/889] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Aug 24 05:25:02 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:25:03.182302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece6e23-f622-4333-99d7-27551e39b931 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:03.198659 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-422292f4-7a06-48d0-898a-0cbc4212060e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:25:03.198659 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:03.198659 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:03.198659 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:03.198659 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:03.198659 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:03.198659 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:03.198659 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:03.198659 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:03.198659 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:03.198659 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:03.198659 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:03.198659 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:25:03.198659 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:25:03.198659 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:25:03.198659 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:25:03.199616 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:03.199616 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:03.199616 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:03.199616 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:03.199616 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:03.199616 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:03.199616 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:03.199616 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:03.199616 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:03.199616 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:03.199616 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:03.199616 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:03.199616 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:03.199616 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:03.199616 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:03.199616 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:03.199616 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:25:03.204514 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:25:03.204514 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:25:03.204514 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:03.204514 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 453/890] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:25:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:25:03.209439 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422292f4-7a06-48d0-898a-0cbc4212060e None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:03.256739 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:03.297391 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece6e23-f622-4333-99d7-27551e39b931 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:03.322522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422292f4-7a06-48d0-898a-0cbc4212060e None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:03.333757 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ea5ae5-ed5b-4c80-89fb-95f1250b12a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:03.334508 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea5ae5-ed5b-4c80-89fb-95f1250b12a5 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:03.334639 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea5ae5-ed5b-4c80-89fb-95f1250b12a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:03.334893 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea5ae5-ed5b-4c80-89fb-95f1250b12a5 None admin] PATH_INFO: `/v3/services` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:03.335520 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ea5ae5-ed5b-4c80-89fb-95f1250b12a5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:03.335818 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ea5ae5-ed5b-4c80-89fb-95f1250b12a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:03.337547 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ea5ae5-ed5b-4c80-89fb-95f1250b12a5 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:03.347514 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 438/891] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Aug 24 05:25:03 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:03.348290 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ea5ae5-ed5b-4c80-89fb-95f1250b12a5 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:03.369528 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc43e7-0bfe-406a-a457-b1b497af30ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:03.370170 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc43e7-0bfe-406a-a457-b1b497af30ee None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:03.370394 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc43e7-0bfe-406a-a457-b1b497af30ee None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:03.370676 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc43e7-0bfe-406a-a457-b1b497af30ee None admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:03.371645 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bc43e7-0bfe-406a-a457-b1b497af30ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:03.371960 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bc43e7-0bfe-406a-a457-b1b497af30ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:03.374095 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bc43e7-0bfe-406a-a457-b1b497af30ee None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:03.394548 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 454/892] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:25:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:03.395730 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc43e7-0bfe-406a-a457-b1b497af30ee None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:03.457496 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:03.470192 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ea5ae5-ed5b-4c80-89fb-95f1250b12a5 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:03.497417 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:03.506265 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba710c3-d49d-4990-9c30-acc1c9502959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:03.506649 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba710c3-d49d-4990-9c30-acc1c9502959 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:03.506928 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba710c3-d49d-4990-9c30-acc1c9502959 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:03.507201 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba710c3-d49d-4990-9c30-acc1c9502959 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:03.507971 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba710c3-d49d-4990-9c30-acc1c9502959 None admin] RBAC: Adding query filter params (interface=public, service_id=df126eaf0e714c9899a1e50d816bd204, region_id=RegionOne) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:03.508231 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba710c3-d49d-4990-9c30-acc1c9502959 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:03.509713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba710c3-d49d-4990-9c30-acc1c9502959 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:03.510431 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc43e7-0bfe-406a-a457-b1b497af30ee None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:03.519582 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 439/893] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Aug 24 05:25:03 2022] GET /identity/v3/endpoints?service_id=df126eaf0e714c9899a1e50d816bd204&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:25:03.520947 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba710c3-d49d-4990-9c30-acc1c9502959 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:03.551029 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9d3236-bb6d-4040-9584-4eea62aca029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:03.551916 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d3236-bb6d-4040-9584-4eea62aca029 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:03.552422 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d3236-bb6d-4040-9584-4eea62aca029 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:03.552872 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d3236-bb6d-4040-9584-4eea62aca029 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:03.562526 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9d3236-bb6d-4040-9584-4eea62aca029 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:03.564221 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9d3236-bb6d-4040-9584-4eea62aca029 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:03.576999 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-1f9d3236-bb6d-4040-9584-4eea62aca029 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 05:25:03.576999 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:03.576999 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:03.576999 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:03.576999 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:03.576999 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:03.576999 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:03.576999 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:03.576999 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:03.576999 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:03.576999 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:03.576999 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:03.576999 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:25:03.576999 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:25:03.576999 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:25:03.576999 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:25:03.577961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:03.577961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:03.577961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:25:03.577961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:25:03.577961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:25:03.577961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:25:03.577961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:03.577961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:03.577961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:03.577961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:03.577961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:03.577961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:03.577961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:03.577961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:03.577961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:03.577961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:03.577961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:03.577961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:03.578882 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:03.578882 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:03.578882 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:25:03.578882 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:25:03.578882 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:25:03.578882 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:25:03.578882 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:25:03.578882 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:25:03.578882 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 05:25:03.578882 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:03.584821 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 455/894] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Aug 24 05:25:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:25:03.587683 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9d3236-bb6d-4040-9584-4eea62aca029 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:03.631096 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba710c3-d49d-4990-9c30-acc1c9502959 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:03.666397 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc202d7-7a09-4c25-a9f3-685551db346d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:03.667009 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc202d7-7a09-4c25-a9f3-685551db346d None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:03.667290 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc202d7-7a09-4c25-a9f3-685551db346d None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:03.667540 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc202d7-7a09-4c25-a9f3-685551db346d None admin] PATH_INFO: `/v3/services` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:03.668141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc202d7-7a09-4c25-a9f3-685551db346d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:03.669680 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc202d7-7a09-4c25-a9f3-685551db346d None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:03.684429 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 440/895] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Aug 24 05:25:03 2022] GET /identity/v3/services => generated 1268 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:25:03.696803 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d37ae-852d-4502-a4c7-ff67c2f9d72b None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:03.697153 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d37ae-852d-4502-a4c7-ff67c2f9d72b None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:03.697365 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d37ae-852d-4502-a4c7-ff67c2f9d72b None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:03.699062 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 441/896] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:25:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:25:03.706583 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb61b57-7662-46f1-bc01-94268fb98a48 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:03.706895 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb61b57-7662-46f1-bc01-94268fb98a48 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:03.707152 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb61b57-7662-46f1-bc01-94268fb98a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:03.723544 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9d3236-bb6d-4040-9584-4eea62aca029 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:03.730434 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-6bb61b57-7662-46f1-bc01-94268fb98a48 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:25:03.732053 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-6bb61b57-7662-46f1-bc01-94268fb98a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:25:03.742940 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:03.768224 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f51d9-36f4-4a41-848f-74c13ed11b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:03.769020 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f51d9-36f4-4a41-848f-74c13ed11b42 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:03.769381 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f51d9-36f4-4a41-848f-74c13ed11b42 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:03.769792 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f51d9-36f4-4a41-848f-74c13ed11b42 None admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:03.770552 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836f51d9-36f4-4a41-848f-74c13ed11b42 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:03.771134 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836f51d9-36f4-4a41-848f-74c13ed11b42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:03.772649 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836f51d9-36f4-4a41-848f-74c13ed11b42 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:03.795088 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 456/897] 10.222.0.15 () {56 vars in 1149 bytes} [Wed Aug 24 05:25:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:25:03.800256 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f51d9-36f4-4a41-848f-74c13ed11b42 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:03.804635 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:03.817423 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 442/898] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:25:03 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:25:03.917391 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f51d9-36f4-4a41-848f-74c13ed11b42 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:03.953723 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ac74f-a41f-42d6-a855-441db9f2deca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:03.954328 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ac74f-a41f-42d6-a855-441db9f2deca None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:03.954607 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ac74f-a41f-42d6-a855-441db9f2deca None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:03.954883 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ac74f-a41f-42d6-a855-441db9f2deca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:03.997256 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:04.024568 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765ac74f-a41f-42d6-a855-441db9f2deca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:04.026023 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765ac74f-a41f-42d6-a855-441db9f2deca None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:04.096708 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb61b57-7662-46f1-bc01-94268fb98a48 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:04.135617 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 457/899] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:25:03 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:25:04.142520 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ac74f-a41f-42d6-a855-441db9f2deca None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:04.231902 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb61b57-7662-46f1-bc01-94268fb98a48 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:04.239890 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:04.273608 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d1e9ab-ea85-4b8c-92a1-41e69b64834d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:04.274242 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d1e9ab-ea85-4b8c-92a1-41e69b64834d None admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:04.274528 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d1e9ab-ea85-4b8c-92a1-41e69b64834d None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:04.274810 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d1e9ab-ea85-4b8c-92a1-41e69b64834d None admin] PATH_INFO: `/v3/services` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:04.275474 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d1e9ab-ea85-4b8c-92a1-41e69b64834d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:04.277185 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d1e9ab-ea85-4b8c-92a1-41e69b64834d None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:04.280722 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ac74f-a41f-42d6-a855-441db9f2deca None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:04.317358 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1981e761-d6ce-41f5-a328-4708d35023c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:04.318071 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981e761-d6ce-41f5-a328-4708d35023c6 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:04.318351 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981e761-d6ce-41f5-a328-4708d35023c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:04.318651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981e761-d6ce-41f5-a328-4708d35023c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:04.334806 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1981e761-d6ce-41f5-a328-4708d35023c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:04.336265 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1981e761-d6ce-41f5-a328-4708d35023c6 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:04.351752 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-1981e761-d6ce-41f5-a328-4708d35023c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:25:04.351752 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:04.351752 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:04.351752 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:04.351752 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:04.351752 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:04.351752 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:04.351752 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:04.351752 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:04.351752 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:04.351752 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:04.351752 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:04.351752 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:25:04.351752 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:25:04.351752 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:25:04.351752 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:25:04.353089 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:04.353089 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:04.353089 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:04.353089 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:04.353089 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:04.353089 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:04.353089 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:04.353089 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:04.353089 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:04.353089 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:04.353089 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:04.353089 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:04.353089 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:04.353089 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:04.353089 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:04.353089 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:04.353089 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:25:04.354604 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:25:04.354604 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:25:04.354604 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:04.357738 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 458/900] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:25:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:25:04.358634 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1981e761-d6ce-41f5-a328-4708d35023c6 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:04.372305 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 443/901] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Aug 24 05:25:04 2022] POST /identity/v3/services => generated 241 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:25:04.479131 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1981e761-d6ce-41f5-a328-4708d35023c6 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:04.506311 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:04.523469 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468ca22-b600-4fef-adf3-23f8bbd2acb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:04.523714 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468ca22-b600-4fef-adf3-23f8bbd2acb1 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:04.523979 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468ca22-b600-4fef-adf3-23f8bbd2acb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:04.524249 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468ca22-b600-4fef-adf3-23f8bbd2acb1 None admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:04.524911 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d468ca22-b600-4fef-adf3-23f8bbd2acb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:04.525211 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d468ca22-b600-4fef-adf3-23f8bbd2acb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:04.526578 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d468ca22-b600-4fef-adf3-23f8bbd2acb1 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:04.539960 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 459/902] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:25:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:04.545672 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d1e9ab-ea85-4b8c-92a1-41e69b64834d None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:04.600328 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:04.658743 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d1e9ab-ea85-4b8c-92a1-41e69b64834d None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:04.693860 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e4463e-3303-41de-bdd1-f5a654d13a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:04.694501 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4463e-3303-41de-bdd1-f5a654d13a08 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:04.694781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4463e-3303-41de-bdd1-f5a654d13a08 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:04.695052 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4463e-3303-41de-bdd1-f5a654d13a08 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:04.702427 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e4463e-3303-41de-bdd1-f5a654d13a08 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:04.703897 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e4463e-3303-41de-bdd1-f5a654d13a08 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:04.754272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-b3e4463e-3303-41de-bdd1-f5a654d13a08 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:25:04.754272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:04.754272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:04.754272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:04.754272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:04.754272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:04.754272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:04.754272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:04.754272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:04.754272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:04.754272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:04.754272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:04.754272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:25:04.754272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:25:04.754272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:25:04.755330 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:25:04.755330 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:04.755330 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:04.755330 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:04.755330 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:04.755330 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:04.755330 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:04.755330 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:04.755330 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:04.755330 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:04.755330 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:04.755330 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:04.755330 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:04.755330 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:04.755330 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:04.755330 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:04.755330 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:04.756243 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:25:04.756243 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:25:04.756243 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:25:04.756243 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:25:04.756243 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:25:04.756243 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:25:04.756243 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:25:04.756243 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:25:04.756243 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 444/903] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:25:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:25:04.756795 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468ca22-b600-4fef-adf3-23f8bbd2acb1 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:04.771090 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358a57a-3174-475c-a93c-4e4c7c4626e8 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:04.771090 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358a57a-3174-475c-a93c-4e4c7c4626e8 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:04.771090 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358a57a-3174-475c-a93c-4e4c7c4626e8 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:04.775551 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 445/904] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:25:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:25:04.777954 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df460f-9fa5-46c0-889d-e3fddbffe05f None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:04.778245 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df460f-9fa5-46c0-889d-e3fddbffe05f None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:04.778527 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df460f-9fa5-46c0-889d-e3fddbffe05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:04.821560 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-a6df460f-9fa5-46c0-889d-e3fddbffe05f None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:25:04.823516 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-a6df460f-9fa5-46c0-889d-e3fddbffe05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:25:04.895741 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468ca22-b600-4fef-adf3-23f8bbd2acb1 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:04.964495 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 446/905] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:25:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:25:04.973182 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3b47e-53d6-4df1-9441-b38d45dfe74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:04.973823 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3b47e-53d6-4df1-9441-b38d45dfe74a None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:04.974095 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3b47e-53d6-4df1-9441-b38d45dfe74a None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:04.974367 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3b47e-53d6-4df1-9441-b38d45dfe74a None admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:04.975072 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3b47e-53d6-4df1-9441-b38d45dfe74a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:04.975379 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3b47e-53d6-4df1-9441-b38d45dfe74a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:04.976946 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3b47e-53d6-4df1-9441-b38d45dfe74a None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:04.992955 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 460/906] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Aug 24 05:25:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:05.004455 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6df460f-9fa5-46c0-889d-e3fddbffe05f None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:05.046447 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:05.118335 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6df460f-9fa5-46c0-889d-e3fddbffe05f None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:05.165686 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c15886-4e8b-46b0-9ef7-81286c6acc1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:05.166885 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c15886-4e8b-46b0-9ef7-81286c6acc1e None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:05.167445 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c15886-4e8b-46b0-9ef7-81286c6acc1e None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:05.167938 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c15886-4e8b-46b0-9ef7-81286c6acc1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:05.168909 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c15886-4e8b-46b0-9ef7-81286c6acc1e None admin] RBAC: Adding query filter params (role.id=719a48d7876b4ffa9489dcc5d87cfbad, scope.project.id=59592706ae9442e08353158c625e9581, user.id=604380c191c8461bbb937b8d951238bc) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:05.170025 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c15886-4e8b-46b0-9ef7-81286c6acc1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:05.172106 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c15886-4e8b-46b0-9ef7-81286c6acc1e None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:05.179724 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 447/907] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:25:05 2022] GET /identity/v3/role_assignments?user.id=604380c191c8461bbb937b8d951238bc&scope.project.id=59592706ae9442e08353158c625e9581&role.id=719a48d7876b4ffa9489dcc5d87cfbad => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:05.281996 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3b47e-53d6-4df1-9441-b38d45dfe74a None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:05.340804 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:05.401650 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3b47e-53d6-4df1-9441-b38d45dfe74a None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:05.438614 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6605175-c623-4d37-9e69-39ff91dca350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:05.439269 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6605175-c623-4d37-9e69-39ff91dca350 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:05.439562 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6605175-c623-4d37-9e69-39ff91dca350 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:05.439845 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6605175-c623-4d37-9e69-39ff91dca350 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:05.440500 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6605175-c623-4d37-9e69-39ff91dca350 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:05.442017 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6605175-c623-4d37-9e69-39ff91dca350 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:05.451374 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-c6605175-c623-4d37-9e69-39ff91dca350 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 05:25:05.451374 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:05.451374 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:25:05.451374 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:25:05.451374 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:25:05.451374 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:25:05.451374 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:25:05.451374 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:25:05.451374 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 05:25:05.451374 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:05.451374 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:25:05.451374 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:05.451374 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:05.451374 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:05.451374 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:05.451374 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:05.452319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:05.452319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:05.452319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:05.452319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:05.452319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:05.452319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:05.452319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:05.452319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:25:05.452319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:25:05.452319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:25:05.452319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:25:05.452319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:05.452319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:05.452319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:05.452319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:05.452319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:05.452319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:05.453224 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:05.453224 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:05.453224 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:05.453224 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:05.453224 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:05.453224 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:05.453224 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:05.453224 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:05.453224 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:05.453224 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:05.453224 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:25:05.453224 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:25:05.453224 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 05:25:05.453224 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:05.457737 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 461/908] 10.222.0.15 () {56 vars in 1132 bytes} [Wed Aug 24 05:25:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:25:05.459133 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c15886-4e8b-46b0-9ef7-81286c6acc1e None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:05.569108 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:05.572984 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c15886-4e8b-46b0-9ef7-81286c6acc1e None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:05.614887 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a9cd4-f605-454d-8441-a132d2ce01ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:05.615619 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a9cd4-f605-454d-8441-a132d2ce01ca None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:05.615904 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a9cd4-f605-454d-8441-a132d2ce01ca None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:05.616183 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a9cd4-f605-454d-8441-a132d2ce01ca None admin] PATH_INFO: `/v3/services` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:05.616965 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588a9cd4-f605-454d-8441-a132d2ce01ca None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:05.617514 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588a9cd4-f605-454d-8441-a132d2ce01ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:05.619002 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588a9cd4-f605-454d-8441-a132d2ce01ca None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:05.629533 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 448/909] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Aug 24 05:25:05 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:05.631815 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6605175-c623-4d37-9e69-39ff91dca350 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:05.740185 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6605175-c623-4d37-9e69-39ff91dca350 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:05.742838 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:05.776818 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d40d6-c01d-4e4c-afb5-1dc7015a9ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:05.777526 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d40d6-c01d-4e4c-afb5-1dc7015a9ae3 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:05.777881 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d40d6-c01d-4e4c-afb5-1dc7015a9ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:05.778228 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d40d6-c01d-4e4c-afb5-1dc7015a9ae3 None admin] PATH_INFO: `/v3/services` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:05.778984 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d40d6-c01d-4e4c-afb5-1dc7015a9ae3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:05.779367 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d40d6-c01d-4e4c-afb5-1dc7015a9ae3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:05.780917 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d40d6-c01d-4e4c-afb5-1dc7015a9ae3 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:05.791446 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 462/910] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Aug 24 05:25:05 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:05.792897 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a9cd4-f605-454d-8441-a132d2ce01ca None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:05.796472 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713637c-5d99-4dd4-bb5f-5963543d9a6e None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:05.796764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713637c-5d99-4dd4-bb5f-5963543d9a6e None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:05.797175 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713637c-5d99-4dd4-bb5f-5963543d9a6e None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:05.798840 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 463/911] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:25:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:25:05.803813 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6cd08-e6c1-422a-95a2-62c6d0d8da84 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:05.804209 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6cd08-e6c1-422a-95a2-62c6d0d8da84 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:05.804527 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6cd08-e6c1-422a-95a2-62c6d0d8da84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:05.824046 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-c1d6cd08-e6c1-422a-95a2-62c6d0d8da84 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:25:05.825284 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-c1d6cd08-e6c1-422a-95a2-62c6d0d8da84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:25:05.906003 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 464/912] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:25:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:25:05.918906 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a9cd4-f605-454d-8441-a132d2ce01ca None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:05.955447 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a8206a-0c79-4daf-be0d-0127f38ab985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:05.956065 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8206a-0c79-4daf-be0d-0127f38ab985 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:05.956352 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8206a-0c79-4daf-be0d-0127f38ab985 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:05.956666 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8206a-0c79-4daf-be0d-0127f38ab985 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:05.957433 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a8206a-0c79-4daf-be0d-0127f38ab985 None admin] RBAC: Adding query filter params (interface=public, service_id=a25f2c6dcf414028b70c67add1f32fd6, region_id=RegionOne) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:05.957748 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a8206a-0c79-4daf-be0d-0127f38ab985 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:05.959249 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a8206a-0c79-4daf-be0d-0127f38ab985 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:05.969841 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 449/913] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Aug 24 05:25:05 2022] GET /identity/v3/endpoints?service_id=a25f2c6dcf414028b70c67add1f32fd6&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:05.973492 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d6cd08-e6c1-422a-95a2-62c6d0d8da84 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:06.074754 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:06.079613 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d6cd08-e6c1-422a-95a2-62c6d0d8da84 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:06.116057 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd60bd-b2cd-4d23-806b-90a1577a5a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:06.116692 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd60bd-b2cd-4d23-806b-90a1577a5a2a None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:06.116971 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd60bd-b2cd-4d23-806b-90a1577a5a2a None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:06.117246 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd60bd-b2cd-4d23-806b-90a1577a5a2a None admin] PATH_INFO: `/v3/services` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:06.117851 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd60bd-b2cd-4d23-806b-90a1577a5a2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:06.119207 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd60bd-b2cd-4d23-806b-90a1577a5a2a None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:06.126165 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 465/914] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Aug 24 05:25:05 2022] GET /identity/v3/services => generated 1497 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:25:06.199354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a8206a-0c79-4daf-be0d-0127f38ab985 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:06.234852 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:06.317053 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a8206a-0c79-4daf-be0d-0127f38ab985 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:06.352800 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506c2e99-5157-4999-aade-2a727a2ceaa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:06.353358 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c2e99-5157-4999-aade-2a727a2ceaa7 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:06.353639 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c2e99-5157-4999-aade-2a727a2ceaa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:06.353917 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c2e99-5157-4999-aade-2a727a2ceaa7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:06.354539 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506c2e99-5157-4999-aade-2a727a2ceaa7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:06.355907 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506c2e99-5157-4999-aade-2a727a2ceaa7 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:06.365836 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-506c2e99-5157-4999-aade-2a727a2ceaa7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 05:25:06.365836 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:06.365836 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:25:06.365836 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:25:06.365836 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:25:06.365836 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:25:06.365836 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:25:06.365836 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:25:06.365836 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 05:25:06.365836 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:25:06.365836 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:25:06.365836 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:25:06.365836 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:06.365836 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:06.365836 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:06.365836 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:06.365836 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:06.366832 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:06.366832 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:06.366832 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:06.366832 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:06.366832 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:06.366832 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:06.366832 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:25:06.366832 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:25:06.366832 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:25:06.366832 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:25:06.366832 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:06.366832 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:06.366832 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:06.366832 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:06.366832 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:06.366832 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:06.367670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:06.367670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:06.367670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:06.367670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:06.367670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:06.367670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:06.367670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:06.367670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:06.367670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:06.367670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:06.367670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:25:06.367670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:25:06.367670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 05:25:06.367670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:25:06.367670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 450/915] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Aug 24 05:25:06 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:25:06.373122 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd60bd-b2cd-4d23-806b-90a1577a5a2a None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:06.476991 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd60bd-b2cd-4d23-806b-90a1577a5a2a None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:06.512990 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc9987-9104-454c-aa78-637bce279be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:06.514265 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc9987-9104-454c-aa78-637bce279be8 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:06.514478 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc9987-9104-454c-aa78-637bce279be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:06.514705 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc9987-9104-454c-aa78-637bce279be8 None admin] PATH_INFO: `/v3/services` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:06.515310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af793d99-27c1-45f9-9417-e4388ceee1c4 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:06.515730 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbc9987-9104-454c-aa78-637bce279be8 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:06.515806 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af793d99-27c1-45f9-9417-e4388ceee1c4 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:06.515888 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbc9987-9104-454c-aa78-637bce279be8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:06.515969 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af793d99-27c1-45f9-9417-e4388ceee1c4 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:06.518079 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbc9987-9104-454c-aa78-637bce279be8 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:06.518217 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 451/916] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:25:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:25:06.523954 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac89dbf-c507-45b5-a33f-510fe7fea6a1 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:06.524533 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac89dbf-c507-45b5-a33f-510fe7fea6a1 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:06.525042 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac89dbf-c507-45b5-a33f-510fe7fea6a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:06.526751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 466/917] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Aug 24 05:25:06 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:06.533190 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc9987-9104-454c-aa78-637bce279be8 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:06.545649 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-3ac89dbf-c507-45b5-a33f-510fe7fea6a1 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:25:06.547325 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-3ac89dbf-c507-45b5-a33f-510fe7fea6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:25:06.568944 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:06.625821 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 452/918] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:25:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:25:06.659216 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc9987-9104-454c-aa78-637bce279be8 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:06.695639 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576a0e4f-4f1e-4b0a-b604-a58726722ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:06.696257 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a0e4f-4f1e-4b0a-b604-a58726722ebe None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:06.696569 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a0e4f-4f1e-4b0a-b604-a58726722ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:06.696847 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a0e4f-4f1e-4b0a-b604-a58726722ebe None admin] PATH_INFO: `/v3/services` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:06.697476 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576a0e4f-4f1e-4b0a-b604-a58726722ebe None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:06.697780 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576a0e4f-4f1e-4b0a-b604-a58726722ebe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:06.699142 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576a0e4f-4f1e-4b0a-b604-a58726722ebe None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:06.705298 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 467/919] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Aug 24 05:25:06 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:06.711015 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac89dbf-c507-45b5-a33f-510fe7fea6a1 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:06.736515 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:06.815563 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac89dbf-c507-45b5-a33f-510fe7fea6a1 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:06.852834 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a28c1d7-79b7-4bb0-ada5-feaac048c74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:06.853786 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28c1d7-79b7-4bb0-ada5-feaac048c74d None admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:06.854255 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28c1d7-79b7-4bb0-ada5-feaac048c74d None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:06.854705 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28c1d7-79b7-4bb0-ada5-feaac048c74d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:06.855558 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a28c1d7-79b7-4bb0-ada5-feaac048c74d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:06.857167 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a28c1d7-79b7-4bb0-ada5-feaac048c74d None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:06.919018 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576a0e4f-4f1e-4b0a-b604-a58726722ebe None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:06.947879 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 453/920] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:25:06 2022] POST /identity/v3/endpoints => generated 346 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:25:07.030733 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:07.041596 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576a0e4f-4f1e-4b0a-b604-a58726722ebe None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:07.094906 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b411f7-6f33-475a-b63c-e7e8c1c6111d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:07.095926 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b411f7-6f33-475a-b63c-e7e8c1c6111d None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:07.096447 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b411f7-6f33-475a-b63c-e7e8c1c6111d None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:07.096958 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b411f7-6f33-475a-b63c-e7e8c1c6111d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:07.097756 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b411f7-6f33-475a-b63c-e7e8c1c6111d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:07.099366 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b411f7-6f33-475a-b63c-e7e8c1c6111d None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:07.111034 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-43b411f7-6f33-475a-b63c-e7e8c1c6111d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:25:07.111034 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:07.111034 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:25:07.111034 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:25:07.111034 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:25:07.111034 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:25:07.111034 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:25:07.111034 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:25:07.111034 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:25:07.111034 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:07.111034 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:25:07.111034 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:07.111034 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:07.111034 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:07.111034 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:07.111034 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:07.111034 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:07.112138 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:07.112138 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:07.112138 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:07.112138 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:07.112138 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:07.112138 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:07.112138 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:25:07.112138 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:25:07.112138 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:25:07.112138 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:25:07.112138 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:07.112138 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:07.112138 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:07.112138 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:07.112138 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:07.112138 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:07.113020 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:07.113020 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:07.113020 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:07.113020 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:07.113020 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:07.113020 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:07.113020 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:07.113020 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:07.113020 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:07.113020 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:07.113020 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:25:07.113020 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:25:07.113020 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:25:07.113020 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:07.115915 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 468/921] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Aug 24 05:25:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:25:07.123160 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a28c1d7-79b7-4bb0-ada5-feaac048c74d None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:07.144148 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8141cc7d-4b78-49c8-b9c2-964dd0ecb829 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:07.144958 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8141cc7d-4b78-49c8-b9c2-964dd0ecb829 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:07.145243 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8141cc7d-4b78-49c8-b9c2-964dd0ecb829 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:07.147179 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 469/922] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:25:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:25:07.152148 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7577838e-29ba-45c1-828e-cdcc844e6fc2 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:07.152467 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7577838e-29ba-45c1-828e-cdcc844e6fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:07.152809 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7577838e-29ba-45c1-828e-cdcc844e6fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:07.179164 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-7577838e-29ba-45c1-828e-cdcc844e6fc2 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:25:07.180258 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-7577838e-29ba-45c1-828e-cdcc844e6fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:25:07.247396 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:07.270206 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 470/923] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:25:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:25:07.282894 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a28c1d7-79b7-4bb0-ada5-feaac048c74d None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:07.323872 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a706070-cf1b-4cd6-841a-6eca80dd09ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:07.324537 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a706070-cf1b-4cd6-841a-6eca80dd09ff None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:07.325003 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a706070-cf1b-4cd6-841a-6eca80dd09ff None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:07.325355 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a706070-cf1b-4cd6-841a-6eca80dd09ff None admin] PATH_INFO: `/v3/services` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:07.327221 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a706070-cf1b-4cd6-841a-6eca80dd09ff None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:07.327588 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a706070-cf1b-4cd6-841a-6eca80dd09ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:07.329202 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a706070-cf1b-4cd6-841a-6eca80dd09ff None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:07.334368 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 454/924] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Aug 24 05:25:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:25:07.340019 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7577838e-29ba-45c1-828e-cdcc844e6fc2 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:07.430011 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:07.452111 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7577838e-29ba-45c1-828e-cdcc844e6fc2 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:07.491404 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabe6e6e-8391-4f7e-916b-2d77c7067fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:07.492022 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe6e6e-8391-4f7e-916b-2d77c7067fac None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:07.492310 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe6e6e-8391-4f7e-916b-2d77c7067fac None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:07.492620 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe6e6e-8391-4f7e-916b-2d77c7067fac None admin] PATH_INFO: `/v3/services` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:07.493257 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabe6e6e-8391-4f7e-916b-2d77c7067fac None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:07.493568 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabe6e6e-8391-4f7e-916b-2d77c7067fac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:07.494998 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabe6e6e-8391-4f7e-916b-2d77c7067fac None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:07.501701 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 471/925] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Aug 24 05:25:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:07.507558 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a706070-cf1b-4cd6-841a-6eca80dd09ff None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:07.599782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabe6e6e-8391-4f7e-916b-2d77c7067fac None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:07.617880 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a706070-cf1b-4cd6-841a-6eca80dd09ff None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:07.664864 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3bc8ee-6afb-402d-bb1f-57c7fba3415e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:07.666007 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3bc8ee-6afb-402d-bb1f-57c7fba3415e None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:07.666903 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3bc8ee-6afb-402d-bb1f-57c7fba3415e None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:07.667705 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3bc8ee-6afb-402d-bb1f-57c7fba3415e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:07.668654 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3bc8ee-6afb-402d-bb1f-57c7fba3415e None admin] RBAC: Adding query filter params (interface=public, service_id=584407fb63a0413fa870f82e2640c7b8, region_id=RegionOne) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:07.669363 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3bc8ee-6afb-402d-bb1f-57c7fba3415e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:07.671836 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3bc8ee-6afb-402d-bb1f-57c7fba3415e None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:07.679542 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 455/926] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Aug 24 05:25:07 2022] GET /identity/v3/endpoints?service_id=584407fb63a0413fa870f82e2640c7b8&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:07.686308 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3bc8ee-6afb-402d-bb1f-57c7fba3415e None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:07.698708 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:07.739954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabe6e6e-8391-4f7e-916b-2d77c7067fac None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:07.781831 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3ea1f8-ccc9-468a-b97d-f373172f5407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:07.782908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ea1f8-ccc9-468a-b97d-f373172f5407 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:07.783460 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ea1f8-ccc9-468a-b97d-f373172f5407 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:07.783953 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ea1f8-ccc9-468a-b97d-f373172f5407 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:07.796929 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3ea1f8-ccc9-468a-b97d-f373172f5407 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:25:07.799187 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3bc8ee-6afb-402d-bb1f-57c7fba3415e None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:07.802559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-cb3ea1f8-ccc9-468a-b97d-f373172f5407 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 05:25:07.802559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:07.802559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:07.802559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:07.802559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:07.802559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:07.802559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:07.802559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:07.802559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:07.802559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:07.802559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:07.802559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:07.802559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:25:07.802559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:25:07.802559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:25:07.802559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:25:07.803486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:25:07.803486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:25:07.803486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:25:07.803486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:25:07.803486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:07.803486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:07.803486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:07.803486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:07.803486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:07.803486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:07.803486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:07.803486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:07.803486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:07.803486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:07.803486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:07.803486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:07.804371 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:07.804371 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:07.804371 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:07.804371 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:07.804371 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:25:07.804371 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:25:07.804371 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:25:07.804371 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:25:07.804371 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:25:07.804371 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:25:07.804371 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 05:25:07.804371 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:07.807211 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 472/927] 10.222.0.15 () {56 vars in 1120 bytes} [Wed Aug 24 05:25:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:25:07.816624 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3ea1f8-ccc9-468a-b97d-f373172f5407 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:07.817991 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:07.836647 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080167e7-ff06-410a-a2a2-7fed98290594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:07.837342 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080167e7-ff06-410a-a2a2-7fed98290594 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:07.837692 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080167e7-ff06-410a-a2a2-7fed98290594 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:07.838035 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080167e7-ff06-410a-a2a2-7fed98290594 None admin] PATH_INFO: `/v3/services` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:07.838784 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080167e7-ff06-410a-a2a2-7fed98290594 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:07.840299 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080167e7-ff06-410a-a2a2-7fed98290594 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:07.847364 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 456/928] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Aug 24 05:25:07 2022] GET /identity/v3/services => generated 1497 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:25:07.933767 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3ea1f8-ccc9-468a-b97d-f373172f5407 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:07.973880 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b8e64c-ad23-4336-8b73-e85d43f2d343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:07.974743 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b8e64c-ad23-4336-8b73-e85d43f2d343 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:07.975188 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b8e64c-ad23-4336-8b73-e85d43f2d343 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:07.975627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b8e64c-ad23-4336-8b73-e85d43f2d343 None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:07.976480 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b8e64c-ad23-4336-8b73-e85d43f2d343 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:07.977001 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b8e64c-ad23-4336-8b73-e85d43f2d343 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:07.978664 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b8e64c-ad23-4336-8b73-e85d43f2d343 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:07.989450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 473/929] 10.222.0.15 () {56 vars in 1115 bytes} [Wed Aug 24 05:25:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:07.995329 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080167e7-ff06-410a-a2a2-7fed98290594 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:08.081499 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:08.114244 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080167e7-ff06-410a-a2a2-7fed98290594 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:08.146518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1ed35-a6c1-407a-8a19-e3559f2f740d None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:08.146831 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1ed35-a6c1-407a-8a19-e3559f2f740d None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:08.147755 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1ed35-a6c1-407a-8a19-e3559f2f740d None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:08.149424 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 474/930] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:25:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:25:08.154079 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e0e68-57c4-44d7-a54a-a8cc99254ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:08.154721 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e0e68-57c4-44d7-a54a-a8cc99254ec4 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:08.155004 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e0e68-57c4-44d7-a54a-a8cc99254ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:08.155280 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e0e68-57c4-44d7-a54a-a8cc99254ec4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:08.160431 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89195ab-f187-4651-91a7-e1d3fdfc1e06 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:08.160727 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89195ab-f187-4651-91a7-e1d3fdfc1e06 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:08.161007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89195ab-f187-4651-91a7-e1d3fdfc1e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:08.168633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15e0e68-57c4-44d7-a54a-a8cc99254ec4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:08.171863 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15e0e68-57c4-44d7-a54a-a8cc99254ec4 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:08.189721 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-c15e0e68-57c4-44d7-a54a-a8cc99254ec4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:25:08.189721 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:08.189721 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:08.189721 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:08.189721 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:08.189721 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:08.189721 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:08.189721 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:08.189721 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:08.189721 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:08.189721 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:08.189721 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:08.189721 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:25:08.189721 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:25:08.189721 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:25:08.189721 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:25:08.190686 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:08.190686 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:08.190686 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:08.190686 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:08.190686 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:08.190686 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:08.190686 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:08.190686 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:08.190686 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:08.190686 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:08.190686 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:08.190686 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:08.190686 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:08.190686 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:08.190686 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:08.190686 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:08.190686 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:25:08.192011 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:25:08.192011 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:25:08.192011 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:25:08.192011 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 457/931] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:25:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:25:08.196855 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-b89195ab-f187-4651-91a7-e1d3fdfc1e06 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:25:08.197044 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e0e68-57c4-44d7-a54a-a8cc99254ec4 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:08.198490 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-b89195ab-f187-4651-91a7-e1d3fdfc1e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:25:08.263615 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:08.271650 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 475/932] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:25:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:25:08.319395 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e0e68-57c4-44d7-a54a-a8cc99254ec4 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:08.343794 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:08.354426 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bf9c72-6e9a-4ca2-a601-a7c7920bc876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:08.355033 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf9c72-6e9a-4ca2-a601-a7c7920bc876 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:08.355312 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf9c72-6e9a-4ca2-a601-a7c7920bc876 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:08.355585 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf9c72-6e9a-4ca2-a601-a7c7920bc876 None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:08.365551 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bf9c72-6e9a-4ca2-a601-a7c7920bc876 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:08.365551 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bf9c72-6e9a-4ca2-a601-a7c7920bc876 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:08.365551 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bf9c72-6e9a-4ca2-a601-a7c7920bc876 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:08.370214 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 458/933] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:25:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:08.376522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89195ab-f187-4651-91a7-e1d3fdfc1e06 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:08.482304 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89195ab-f187-4651-91a7-e1d3fdfc1e06 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:08.518156 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f87d53-a69e-4dc8-a001-41321f0dba9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:08.518719 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f87d53-a69e-4dc8-a001-41321f0dba9b None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:08.518998 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f87d53-a69e-4dc8-a001-41321f0dba9b None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:08.519270 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f87d53-a69e-4dc8-a001-41321f0dba9b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:08.526837 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f87d53-a69e-4dc8-a001-41321f0dba9b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:08.528283 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f87d53-a69e-4dc8-a001-41321f0dba9b None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:08.540636 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-93f87d53-a69e-4dc8-a001-41321f0dba9b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 05:25:08.540636 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:08.540636 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:08.540636 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:08.540636 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:08.540636 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:08.540636 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:08.540636 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:08.540636 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:08.540636 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:08.540636 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:08.540636 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:08.540636 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:25:08.540636 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:25:08.540636 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:25:08.540636 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:25:08.541614 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:08.541614 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:08.541614 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:25:08.541614 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:25:08.541614 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:25:08.541614 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:25:08.541614 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:08.541614 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:08.541614 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:08.541614 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:08.541614 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:08.541614 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:08.541614 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:08.541614 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:08.541614 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:08.541614 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:08.541614 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:08.541614 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:08.543136 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:08.543136 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:08.543136 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:25:08.543136 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:25:08.543136 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:25:08.543136 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:25:08.543136 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:25:08.543136 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:25:08.543136 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 05:25:08.543136 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:08.543136 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 476/934] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Aug 24 05:25:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:25:08.548378 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bf9c72-6e9a-4ca2-a601-a7c7920bc876 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:08.553762 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f87d53-a69e-4dc8-a001-41321f0dba9b None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:08.620729 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:08.671364 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f87d53-a69e-4dc8-a001-41321f0dba9b None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:08.688684 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bf9c72-6e9a-4ca2-a601-a7c7920bc876 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:08.707907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538189ea-6147-47d7-b218-0913f3363f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:08.708845 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538189ea-6147-47d7-b218-0913f3363f27 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:08.708956 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538189ea-6147-47d7-b218-0913f3363f27 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:08.709150 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538189ea-6147-47d7-b218-0913f3363f27 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:08.709789 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538189ea-6147-47d7-b218-0913f3363f27 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:08.711216 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538189ea-6147-47d7-b218-0913f3363f27 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:08.720500 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-538189ea-6147-47d7-b218-0913f3363f27 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 05:25:08.720500 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:08.720500 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:25:08.720500 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:25:08.720500 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:25:08.720500 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:25:08.720500 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:25:08.720500 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:25:08.720500 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 05:25:08.720500 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:08.720500 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:25:08.720500 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:08.720500 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:08.720500 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:08.720500 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:08.720500 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:08.721450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:08.721450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:08.721450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:08.721450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:08.721450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:08.721450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:08.721450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:08.721450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:25:08.721450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:25:08.721450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:25:08.721450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:25:08.721450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:08.721450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:08.721450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:08.721450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:08.721450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:08.721450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:08.727274 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:08.727274 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:08.727274 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:08.727274 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:08.727274 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:08.727274 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:08.727274 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:08.727274 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:08.727274 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:08.727274 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:08.727274 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:25:08.727274 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:25:08.727274 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 05:25:08.727274 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:08.727274 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 477/935] 10.222.0.15 () {56 vars in 1132 bytes} [Wed Aug 24 05:25:08 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:25:08.729464 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538189ea-6147-47d7-b218-0913f3363f27 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:08.738740 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a99701-38ec-4138-8d7a-d456e33eab37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:08.739451 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a99701-38ec-4138-8d7a-d456e33eab37 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:08.739801 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a99701-38ec-4138-8d7a-d456e33eab37 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:08.740143 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a99701-38ec-4138-8d7a-d456e33eab37 None admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:08.741142 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a99701-38ec-4138-8d7a-d456e33eab37 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:08.741526 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a99701-38ec-4138-8d7a-d456e33eab37 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:08.743117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a99701-38ec-4138-8d7a-d456e33eab37 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:08.767452 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 459/936] 10.222.0.15 () {56 vars in 1149 bytes} [Wed Aug 24 05:25:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:08.768624 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a99701-38ec-4138-8d7a-d456e33eab37 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:08.819033 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:08.844433 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538189ea-6147-47d7-b218-0913f3363f27 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:08.866110 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:08.883087 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61c62ea-0773-472b-b282-012ccf33bc08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:08.885877 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c62ea-0773-472b-b282-012ccf33bc08 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:08.885877 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c62ea-0773-472b-b282-012ccf33bc08 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:08.885877 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c62ea-0773-472b-b282-012ccf33bc08 None admin] PATH_INFO: `/v3/services` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:08.885877 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61c62ea-0773-472b-b282-012ccf33bc08 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:08.886188 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a99701-38ec-4138-8d7a-d456e33eab37 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:08.886268 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61c62ea-0773-472b-b282-012ccf33bc08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:08.887182 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61c62ea-0773-472b-b282-012ccf33bc08 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:08.897629 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 478/937] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Aug 24 05:25:08 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:08.901122 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61c62ea-0773-472b-b282-012ccf33bc08 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:08.933182 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ebd8e-894f-4f2c-860b-6e7685cf86ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:08.933928 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ebd8e-894f-4f2c-860b-6e7685cf86ee None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:08.934213 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ebd8e-894f-4f2c-860b-6e7685cf86ee None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:08.934671 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ebd8e-894f-4f2c-860b-6e7685cf86ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:08.948602 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510ebd8e-894f-4f2c-860b-6e7685cf86ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:08.950935 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510ebd8e-894f-4f2c-860b-6e7685cf86ee None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:08.970043 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-510ebd8e-894f-4f2c-860b-6e7685cf86ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:25:08.970043 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:08.970043 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:08.970043 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:08.970043 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:08.970043 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:08.970043 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:08.970043 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:08.970043 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:08.970043 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:08.970043 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:08.970043 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:08.970043 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:25:08.970043 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:25:08.970043 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:25:08.970043 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:25:08.971066 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:08.971066 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:08.971066 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:08.971066 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:08.971066 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:08.971066 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:08.971066 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:08.971066 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:08.971066 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:08.971066 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:08.971066 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:08.971066 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:08.971066 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:08.971066 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:08.971066 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:08.971066 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:08.971066 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:25:08.971948 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:25:08.971948 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:25:08.971948 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:25:08.976345 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 460/938] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:25:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:25:08.977231 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ebd8e-894f-4f2c-860b-6e7685cf86ee None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:09.039651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61c62ea-0773-472b-b282-012ccf33bc08 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:09.079280 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239e4b2-8e1c-4800-a25c-6c2742fe48de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:09.079894 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239e4b2-8e1c-4800-a25c-6c2742fe48de None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:09.080166 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239e4b2-8e1c-4800-a25c-6c2742fe48de None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:09.080653 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239e4b2-8e1c-4800-a25c-6c2742fe48de None admin] PATH_INFO: `/v3/services` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:09.081458 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d239e4b2-8e1c-4800-a25c-6c2742fe48de None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:09.081764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d239e4b2-8e1c-4800-a25c-6c2742fe48de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:09.089569 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d239e4b2-8e1c-4800-a25c-6c2742fe48de None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:09.095428 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:09.096979 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 479/939] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Aug 24 05:25:08 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:09.098206 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c83d60-db69-4968-bbc2-fd7d349bb6cd None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:09.098493 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c83d60-db69-4968-bbc2-fd7d349bb6cd None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:09.098775 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c83d60-db69-4968-bbc2-fd7d349bb6cd None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:09.100166 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 480/940] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:25:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:25:09.105013 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c83d60-db69-4968-bbc2-fd7d349bb6cd None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:09.110774 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ebd8e-894f-4f2c-860b-6e7685cf86ee None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:09.146379 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c949ae-66de-4152-a1ae-00c89ce992ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:09.147035 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c949ae-66de-4152-a1ae-00c89ce992ba None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:09.147330 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c949ae-66de-4152-a1ae-00c89ce992ba None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:09.147602 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c949ae-66de-4152-a1ae-00c89ce992ba None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:09.148238 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c949ae-66de-4152-a1ae-00c89ce992ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:09.148571 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c949ae-66de-4152-a1ae-00c89ce992ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:09.150053 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c949ae-66de-4152-a1ae-00c89ce992ba None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:09.159244 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:09.167301 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 461/941] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:25:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:09.172353 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e347c-70d5-427e-bc9f-6a4a73b7593c None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:09.172353 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e347c-70d5-427e-bc9f-6a4a73b7593c None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:09.172353 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e347c-70d5-427e-bc9f-6a4a73b7593c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:09.190258 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-176e347c-70d5-427e-bc9f-6a4a73b7593c None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:25:09.191446 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-176e347c-70d5-427e-bc9f-6a4a73b7593c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:25:09.228298 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c83d60-db69-4968-bbc2-fd7d349bb6cd None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:09.266779 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 462/942] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:25:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:25:09.267991 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e347c-70d5-427e-bc9f-6a4a73b7593c None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:09.279228 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eee10b-eb49-4d45-9379-5e1daf76c07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:09.280143 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eee10b-eb49-4d45-9379-5e1daf76c07f None admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:09.280695 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eee10b-eb49-4d45-9379-5e1daf76c07f None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:09.281405 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eee10b-eb49-4d45-9379-5e1daf76c07f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:09.282244 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eee10b-eb49-4d45-9379-5e1daf76c07f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:09.283959 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eee10b-eb49-4d45-9379-5e1daf76c07f None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:09.361176 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:09.380291 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e347c-70d5-427e-bc9f-6a4a73b7593c None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:09.417690 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b635538b-1fca-4586-853c-21de98d7212a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:09.418385 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635538b-1fca-4586-853c-21de98d7212a None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:09.418673 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635538b-1fca-4586-853c-21de98d7212a None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:09.418950 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635538b-1fca-4586-853c-21de98d7212a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:09.426813 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b635538b-1fca-4586-853c-21de98d7212a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:09.428263 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b635538b-1fca-4586-853c-21de98d7212a None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:09.440633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-b635538b-1fca-4586-853c-21de98d7212a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:25:09.440633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:09.440633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:09.440633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:09.440633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:09.440633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:09.440633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:09.440633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:09.440633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:09.440633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:09.440633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:09.440633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:09.440633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:25:09.440633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:25:09.440633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:25:09.441553 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:25:09.441553 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:09.441553 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:09.441553 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:09.441553 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:09.441553 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:09.441553 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:09.441553 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:09.441553 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:09.441553 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:09.441553 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:09.441553 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:09.441553 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:09.441553 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:09.441553 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:09.441553 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:09.441553 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:09.442450 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:25:09.442450 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:25:09.442450 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:25:09.442450 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:25:09.442450 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:25:09.442450 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:25:09.442450 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:25:09.442450 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:25:09.446733 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 463/943] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:25:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:25:09.448210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b635538b-1fca-4586-853c-21de98d7212a None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:09.454896 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 481/944] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:25:09 2022] POST /identity/v3/endpoints => generated 347 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:25:09.541611 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eee10b-eb49-4d45-9379-5e1daf76c07f None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:09.580473 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b635538b-1fca-4586-853c-21de98d7212a None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:09.630396 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd8b437-63d1-4a7c-a217-54fbce63ea7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:09.631273 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd8b437-63d1-4a7c-a217-54fbce63ea7a None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:09.631727 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd8b437-63d1-4a7c-a217-54fbce63ea7a None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:09.632248 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd8b437-63d1-4a7c-a217-54fbce63ea7a None admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:09.632981 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd8b437-63d1-4a7c-a217-54fbce63ea7a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:09.633291 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd8b437-63d1-4a7c-a217-54fbce63ea7a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:09.634738 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd8b437-63d1-4a7c-a217-54fbce63ea7a None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:09.645615 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:09.652540 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 464/945] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Aug 24 05:25:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:25:09.660164 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd8b437-63d1-4a7c-a217-54fbce63ea7a None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:09.675234 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eee10b-eb49-4d45-9379-5e1daf76c07f None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:09.679000 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:09.719933 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979fcd32-f24d-44d7-b264-f6eeb344fa37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:09.720600 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979fcd32-f24d-44d7-b264-f6eeb344fa37 None admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:09.720890 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979fcd32-f24d-44d7-b264-f6eeb344fa37 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:09.721171 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979fcd32-f24d-44d7-b264-f6eeb344fa37 None admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:09.721821 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979fcd32-f24d-44d7-b264-f6eeb344fa37 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:09.723274 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979fcd32-f24d-44d7-b264-f6eeb344fa37 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:09.726185 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: WARNING keystone.server.flask.common [None req-979fcd32-f24d-44d7-b264-f6eeb344fa37 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 05:25:09.784971 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.sql.core [None req-979fcd32-f24d-44d7-b264-f6eeb344fa37 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 05:25:09.784971 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 05:25:09.784971 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [parameters: {'id': '7e5015a4708b4b8a83e9599393e3f5b1', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 05:25:09.784971 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81670) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 05:25:09.797297 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd8b437-63d1-4a7c-a217-54fbce63ea7a None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:09.807576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-979fcd32-f24d-44d7-b264-f6eeb344fa37 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7e5015a4708b4b8a83e9599393e3f5b1.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7e5015a4708b4b8a83e9599393e3f5b1. Aug 24 05:25:09.807576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:09.807576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 05:25:09.807576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 05:25:09.807576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 05:25:09.807576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 05:25:09.807576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 05:25:09.807576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 05:25:09.807576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 05:25:09.807576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application conn.query(q) Aug 24 05:25:09.807576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 05:25:09.807576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 05:25:09.807576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 05:25:09.808522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application result.read() Aug 24 05:25:09.808522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 05:25:09.808522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 05:25:09.808522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 05:25:09.808522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 05:25:09.808522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 05:25:09.808522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 05:25:09.808522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 05:25:09.808522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 05:25:09.808522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 05:25:09.808522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:09.808522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 05:25:09.808522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:09.808522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:09.808522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 05:25:09.808522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 05:25:09.808522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 24 05:25:09.808522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 24 05:25:09.809479 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 05:25:09.809479 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application next(self.gen) Aug 24 05:25:09.809479 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 05:25:09.809479 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application yield resource Aug 24 05:25:09.809479 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 05:25:09.809479 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application next(self.gen) Aug 24 05:25:09.809479 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 05:25:09.809479 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 05:25:09.809479 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 05:25:09.809479 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 05:25:09.809479 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 05:25:09.809479 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise self.value Aug 24 05:25:09.809479 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 05:25:09.809479 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 05:25:09.809479 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 05:25:09.809479 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application session.commit() Aug 24 05:25:09.809479 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 05:25:09.810399 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 05:25:09.810399 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 05:25:09.810399 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 05:25:09.810399 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 05:25:09.810399 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application self.session.flush() Aug 24 05:25:09.810399 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 05:25:09.810399 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 05:25:09.810399 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 05:25:09.810399 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 05:25:09.810399 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 05:25:09.810399 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application compat.raise_( Aug 24 05:25:09.810399 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 05:25:09.810399 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception Aug 24 05:25:09.810399 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 05:25:09.810399 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 05:25:09.810399 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 05:25:09.810399 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 05:25:09.811402 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 05:25:09.811402 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 05:25:09.811402 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 05:25:09.811402 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 05:25:09.811402 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 05:25:09.811402 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 05:25:09.811402 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 05:25:09.811402 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 05:25:09.811402 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 05:25:09.811402 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 05:25:09.811402 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 05:25:09.811402 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 05:25:09.811402 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 05:25:09.811402 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 05:25:09.811402 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 05:25:09.812187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 05:25:09.812187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 05:25:09.812187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception Aug 24 05:25:09.812187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 05:25:09.812187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 05:25:09.812187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 05:25:09.812187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 05:25:09.812187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 05:25:09.812187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 05:25:09.812187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 05:25:09.812187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application conn.query(q) Aug 24 05:25:09.812187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 05:25:09.812187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 05:25:09.812187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 05:25:09.812187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application result.read() Aug 24 05:25:09.812187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 05:25:09.812187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 05:25:09.813107 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 05:25:09.813107 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 05:25:09.813107 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 05:25:09.813107 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 05:25:09.813107 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 05:25:09.813107 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 05:25:09.813107 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 05:25:09.813107 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 05:25:09.813107 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [parameters: {'id': '7e5015a4708b4b8a83e9599393e3f5b1', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 05:25:09.813107 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 05:25:09.813107 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:09.813107 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:25:09.813107 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:09.813107 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:09.813107 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 24 05:25:09.813897 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 24 05:25:09.813897 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 05:25:09.813897 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 05:25:09.813897 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 24 05:25:09.813897 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:09.813897 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:25:09.813897 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:09.813897 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:09.813897 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:09.813897 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:09.813897 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:09.813897 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:09.813897 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:09.813897 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:09.813897 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:09.813897 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:09.813897 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:09.814820 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:09.814820 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 24 05:25:09.814820 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 24 05:25:09.814820 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:09.814820 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:09.814820 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 24 05:25:09.814820 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 24 05:25:09.814820 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7e5015a4708b4b8a83e9599393e3f5b1. Aug 24 05:25:09.814820 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:09.814820 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 482/946] 10.222.0.15 () {60 vars in 1140 bytes} [Wed Aug 24 05:25:09 2022] POST /identity/v3/projects => generated 262 bytes in 268 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:25:09.817220 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979fcd32-f24d-44d7-b264-f6eeb344fa37 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:09.848521 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12f4add-238c-4142-8c82-d07d7fdebbe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:09.849201 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f4add-238c-4142-8c82-d07d7fdebbe4 None admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:09.849479 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f4add-238c-4142-8c82-d07d7fdebbe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:09.849753 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f4add-238c-4142-8c82-d07d7fdebbe4 None admin] PATH_INFO: `/v3/projects/59592706ae9442e08353158c625e9581/users/604380c191c8461bbb937b8d951238bc/roles/719a48d7876b4ffa9489dcc5d87cfbad` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:09.852838 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12f4add-238c-4142-8c82-d07d7fdebbe4 None admin] RBAC: Authorizing `identity:create_grant(project_id=59592706ae9442e08353158c625e9581, user_id=604380c191c8461bbb937b8d951238bc, role_id=719a48d7876b4ffa9489dcc5d87cfbad)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:09.855401 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12f4add-238c-4142-8c82-d07d7fdebbe4 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:09.929967 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979fcd32-f24d-44d7-b264-f6eeb344fa37 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:09.964188 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17172891-c7df-4edc-9e20-955f18bf65a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:09.965130 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17172891-c7df-4edc-9e20-955f18bf65a0 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:09.965407 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17172891-c7df-4edc-9e20-955f18bf65a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:09.965767 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17172891-c7df-4edc-9e20-955f18bf65a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:09.972499 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 465/947] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Aug 24 05:25:09 2022] PUT /identity/v3/projects/59592706ae9442e08353158c625e9581/users/604380c191c8461bbb937b8d951238bc/roles/719a48d7876b4ffa9489dcc5d87cfbad => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:25:09.973435 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17172891-c7df-4edc-9e20-955f18bf65a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:09.975121 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17172891-c7df-4edc-9e20-955f18bf65a0 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:09.982122 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5dc022-6ef8-44ac-a13f-af6623b7c32b None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:09.982122 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5dc022-6ef8-44ac-a13f-af6623b7c32b None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:09.982122 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5dc022-6ef8-44ac-a13f-af6623b7c32b None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:09.984194 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 466/948] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:25:09 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:25:09.991703 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-17172891-c7df-4edc-9e20-955f18bf65a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:25:09.991703 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:09.991703 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:09.991703 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:09.991703 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:09.991703 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:09.991703 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:09.991703 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:09.991703 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:09.991703 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:09.991703 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:09.991703 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:09.991703 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:25:09.991703 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:25:09.991703 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:25:09.995488 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:25:09.995488 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:09.995488 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:09.995488 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:09.995488 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:09.995488 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:09.995488 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:09.995488 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:09.995488 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:09.995488 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:09.995488 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:09.995488 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:09.995488 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:09.995488 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:09.995488 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:09.995488 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:09.995488 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:09.996389 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:25:09.996389 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:25:09.996389 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:25:09.996389 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:25:09.996389 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:25:09.996389 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:25:09.996389 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:25:09.996389 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:09.996389 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 483/949] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:25:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:25:09.996927 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac329292-c344-4411-87bf-0fd2ff2aa42a None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:09.996927 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac329292-c344-4411-87bf-0fd2ff2aa42a None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:09.996927 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac329292-c344-4411-87bf-0fd2ff2aa42a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:10.000612 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17172891-c7df-4edc-9e20-955f18bf65a0 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:10.029136 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:10.037623 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-ac329292-c344-4411-87bf-0fd2ff2aa42a None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:25:10.040187 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-ac329292-c344-4411-87bf-0fd2ff2aa42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:25:10.122194 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:10.130075 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17172891-c7df-4edc-9e20-955f18bf65a0 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:10.143008 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 467/950] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:25:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:25:10.172931 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d405700-0c33-464d-8c94-1f4279a630c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:10.173701 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d405700-0c33-464d-8c94-1f4279a630c8 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:10.173990 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d405700-0c33-464d-8c94-1f4279a630c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:10.174268 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d405700-0c33-464d-8c94-1f4279a630c8 None admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:10.174943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d405700-0c33-464d-8c94-1f4279a630c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:10.175253 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d405700-0c33-464d-8c94-1f4279a630c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:10.178923 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d405700-0c33-464d-8c94-1f4279a630c8 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:10.203239 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 484/951] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Aug 24 05:25:09 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:25:10.420573 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac329292-c344-4411-87bf-0fd2ff2aa42a None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:10.530298 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac329292-c344-4411-87bf-0fd2ff2aa42a None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:10.568238 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfcc091-59a1-436e-9b4f-41befafbb55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:10.569279 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfcc091-59a1-436e-9b4f-41befafbb55b None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:10.569765 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfcc091-59a1-436e-9b4f-41befafbb55b None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:10.570261 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfcc091-59a1-436e-9b4f-41befafbb55b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:10.571074 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfcc091-59a1-436e-9b4f-41befafbb55b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:10.572681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfcc091-59a1-436e-9b4f-41befafbb55b None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:10.582734 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-fbfcc091-59a1-436e-9b4f-41befafbb55b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:25:10.582734 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:10.582734 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:25:10.582734 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:25:10.582734 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:25:10.582734 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:25:10.582734 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:25:10.582734 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:25:10.582734 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:25:10.582734 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:25:10.582734 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:25:10.582734 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:25:10.582734 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:10.582734 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:10.582734 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:10.582734 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:10.582734 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:10.583901 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:10.583901 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:10.583901 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:10.583901 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:10.583901 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:10.583901 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:10.583901 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:25:10.583901 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:25:10.583901 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:25:10.583901 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:25:10.583901 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:10.583901 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:10.583901 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:10.583901 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:10.583901 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:10.583901 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:10.584853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:10.584853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:10.584853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:10.584853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:10.584853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:10.584853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:10.584853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:10.584853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:10.584853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:10.584853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:10.584853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:25:10.584853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:25:10.584853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:25:10.584853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:25:10.587712 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 468/952] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Aug 24 05:25:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:25:10.593827 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d405700-0c33-464d-8c94-1f4279a630c8 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:10.680881 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:10.702110 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d405700-0c33-464d-8c94-1f4279a630c8 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:10.742941 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d7e423-5491-483c-adc3-3b6c71924110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:10.743555 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d7e423-5491-483c-adc3-3b6c71924110 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:10.743833 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d7e423-5491-483c-adc3-3b6c71924110 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:10.744107 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d7e423-5491-483c-adc3-3b6c71924110 None admin] PATH_INFO: `/v3/services` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:10.744770 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d7e423-5491-483c-adc3-3b6c71924110 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:10.745075 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d7e423-5491-483c-adc3-3b6c71924110 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:10.746446 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d7e423-5491-483c-adc3-3b6c71924110 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:10.752599 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 485/953] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Aug 24 05:25:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:10.759626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfcc091-59a1-436e-9b4f-41befafbb55b None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:10.822112 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:10.873378 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfcc091-59a1-436e-9b4f-41befafbb55b None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:10.907717 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fbfa6c-5537-4a9f-ac33-4b3ddb48b85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:10.908326 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbfa6c-5537-4a9f-ac33-4b3ddb48b85c None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:10.908635 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbfa6c-5537-4a9f-ac33-4b3ddb48b85c None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:10.908907 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbfa6c-5537-4a9f-ac33-4b3ddb48b85c None admin] PATH_INFO: `/v3/services` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:10.909531 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fbfa6c-5537-4a9f-ac33-4b3ddb48b85c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:10.909833 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fbfa6c-5537-4a9f-ac33-4b3ddb48b85c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:10.911182 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fbfa6c-5537-4a9f-ac33-4b3ddb48b85c None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:10.916007 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 469/954] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Aug 24 05:25:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:25:10.925581 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d7e423-5491-483c-adc3-3b6c71924110 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:11.032771 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d7e423-5491-483c-adc3-3b6c71924110 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:11.071640 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7810ffa-6ef4-4317-af1d-87a74cca4977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:11.072319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7810ffa-6ef4-4317-af1d-87a74cca4977 None admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:11.072576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7810ffa-6ef4-4317-af1d-87a74cca4977 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:11.072853 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7810ffa-6ef4-4317-af1d-87a74cca4977 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:11.073527 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7810ffa-6ef4-4317-af1d-87a74cca4977 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:11.077029 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7810ffa-6ef4-4317-af1d-87a74cca4977 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:11.151522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:11.159732 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 486/955] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:25:10 2022] POST /identity/v3/endpoints => generated 349 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:25:11.596019 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3102fdb-88aa-42b6-8cb7-d79bfa02297b None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:11.596360 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3102fdb-88aa-42b6-8cb7-d79bfa02297b None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:11.596631 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3102fdb-88aa-42b6-8cb7-d79bfa02297b None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:11.598227 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 470/956] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:25:11.603695 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799cd25f-5acc-436f-b25e-b3a9e5a302e5 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:11.603989 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799cd25f-5acc-436f-b25e-b3a9e5a302e5 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:11.604270 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799cd25f-5acc-436f-b25e-b3a9e5a302e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:11.625115 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-799cd25f-5acc-436f-b25e-b3a9e5a302e5 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:25:11.626680 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-799cd25f-5acc-436f-b25e-b3a9e5a302e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:25:11.709164 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 487/957] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:25:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:25:11.980547 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3102fdb-88aa-42b6-8cb7-d79bfa02297b None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:12.068009 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d850ee-18e7-4be7-8fbe-4251866ddc0c None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:12.068548 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d850ee-18e7-4be7-8fbe-4251866ddc0c None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:12.068625 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d850ee-18e7-4be7-8fbe-4251866ddc0c None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:12.070486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 488/958] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:25:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:25:12.075975 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990e722-ff79-4159-b463-09cd54c7f4be None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:12.076312 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990e722-ff79-4159-b463-09cd54c7f4be None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:12.076658 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990e722-ff79-4159-b463-09cd54c7f4be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:12.090877 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3102fdb-88aa-42b6-8cb7-d79bfa02297b None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:12.098581 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-3990e722-ff79-4159-b463-09cd54c7f4be None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:25:12.100576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-3990e722-ff79-4159-b463-09cd54c7f4be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:25:12.109354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:12.127630 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344137b-b5e5-4d9f-914a-ba532b77bec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:12.128261 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344137b-b5e5-4d9f-914a-ba532b77bec0 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:12.128593 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344137b-b5e5-4d9f-914a-ba532b77bec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:12.128867 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344137b-b5e5-4d9f-914a-ba532b77bec0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:12.130309 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1344137b-b5e5-4d9f-914a-ba532b77bec0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:12.130858 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1344137b-b5e5-4d9f-914a-ba532b77bec0 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:12.141155 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-1344137b-b5e5-4d9f-914a-ba532b77bec0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 05:25:12.141155 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:12.141155 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:25:12.141155 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:25:12.141155 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:25:12.141155 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:25:12.141155 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:25:12.141155 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:25:12.141155 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 05:25:12.141155 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:25:12.141155 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:25:12.141155 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:25:12.141155 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:12.141155 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:12.141155 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:12.141155 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:12.141155 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:12.142166 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:12.142166 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:12.142166 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:12.142166 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:12.142166 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:12.142166 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:12.142166 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:25:12.142166 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:25:12.142166 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:25:12.142166 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:25:12.142166 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:12.142166 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:12.142166 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:12.142166 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:12.142166 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:12.142166 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:12.142992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:12.142992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:12.142992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:12.142992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:12.142992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:12.142992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:12.142992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:12.142992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:12.142992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:12.142992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:12.142992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:25:12.142992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:25:12.142992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 05:25:12.142992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:25:12.142992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 471/959] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:25:11 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:25:12.149631 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344137b-b5e5-4d9f-914a-ba532b77bec0 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:12.202203 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 489/960] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:25:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:25:12.269118 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344137b-b5e5-4d9f-914a-ba532b77bec0 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:12.308158 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4fdd63-8699-43dc-b789-10f94796275c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:12.309043 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4fdd63-8699-43dc-b789-10f94796275c None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:12.309529 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4fdd63-8699-43dc-b789-10f94796275c None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:12.310112 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4fdd63-8699-43dc-b789-10f94796275c None admin] PATH_INFO: `/v3/services` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:12.311140 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4fdd63-8699-43dc-b789-10f94796275c None admin] RBAC: Adding query filter params (name=compute) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:12.311608 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4fdd63-8699-43dc-b789-10f94796275c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:12.313138 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4fdd63-8699-43dc-b789-10f94796275c None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:12.319076 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 472/961] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Aug 24 05:25:12 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:25:12.324556 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990e722-ff79-4159-b463-09cd54c7f4be None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:12.386513 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:12.407205 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6fbea0-efc3-4db5-bc1d-725463bd22be None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:12.407609 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6fbea0-efc3-4db5-bc1d-725463bd22be None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:12.407897 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6fbea0-efc3-4db5-bc1d-725463bd22be None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:12.409278 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 473/962] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:25:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:25:12.414286 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43765b1c-30cd-4212-aeca-632ae58c52af None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:12.414580 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43765b1c-30cd-4212-aeca-632ae58c52af None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:12.414866 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43765b1c-30cd-4212-aeca-632ae58c52af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:12.434738 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990e722-ff79-4159-b463-09cd54c7f4be None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:12.436343 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-43765b1c-30cd-4212-aeca-632ae58c52af None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:25:12.437813 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-43765b1c-30cd-4212-aeca-632ae58c52af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:25:12.483842 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:12.484417 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc65a7-1ffc-47e5-8511-40bca5ebd076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:12.485058 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc65a7-1ffc-47e5-8511-40bca5ebd076 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:12.485338 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc65a7-1ffc-47e5-8511-40bca5ebd076 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:12.485617 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc65a7-1ffc-47e5-8511-40bca5ebd076 None admin] PATH_INFO: `/v3/services` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:12.486256 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dc65a7-1ffc-47e5-8511-40bca5ebd076 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:12.486563 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dc65a7-1ffc-47e5-8511-40bca5ebd076 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:12.487929 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dc65a7-1ffc-47e5-8511-40bca5ebd076 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:12.492901 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 490/963] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Aug 24 05:25:12 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:12.520876 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 474/964] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:25:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:25:12.525450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc65a7-1ffc-47e5-8511-40bca5ebd076 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:12.639122 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc65a7-1ffc-47e5-8511-40bca5ebd076 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:12.663289 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:12.677655 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf57a9d-524e-4abf-9e59-36761faace7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:12.678299 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf57a9d-524e-4abf-9e59-36761faace7f None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:12.678580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf57a9d-524e-4abf-9e59-36761faace7f None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:12.678856 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf57a9d-524e-4abf-9e59-36761faace7f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:12.690464 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf57a9d-524e-4abf-9e59-36761faace7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:25:12.695195 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-0cf57a9d-524e-4abf-9e59-36761faace7f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 05:25:12.695195 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:12.695195 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:12.695195 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:12.695195 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:12.695195 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:12.695195 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:12.695195 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:12.695195 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:12.695195 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:12.695195 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:12.695195 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:12.695195 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:25:12.695195 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:25:12.695195 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:25:12.695195 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:25:12.696318 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:25:12.696318 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:25:12.696318 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:25:12.696318 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:25:12.696318 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:12.696318 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:12.696318 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:12.696318 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:12.696318 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:12.696318 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:12.696318 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:12.696318 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:12.696318 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:12.696318 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:12.696318 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:12.696318 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:12.697250 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:12.697250 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:12.697250 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:12.697250 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:12.697250 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:25:12.697250 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:25:12.697250 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:25:12.697250 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:25:12.697250 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:25:12.697250 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:25:12.697250 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 05:25:12.697250 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:12.701998 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 491/965] 10.222.0.15 () {56 vars in 1120 bytes} [Wed Aug 24 05:25:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:25:12.707822 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43765b1c-30cd-4212-aeca-632ae58c52af None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:12.823317 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43765b1c-30cd-4212-aeca-632ae58c52af None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:12.823491 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf57a9d-524e-4abf-9e59-36761faace7f None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:12.866325 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc402aa-7330-4fbf-bbad-a4da5ab7e8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:12.866934 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc402aa-7330-4fbf-bbad-a4da5ab7e8c7 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:12.867217 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc402aa-7330-4fbf-bbad-a4da5ab7e8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:12.867494 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc402aa-7330-4fbf-bbad-a4da5ab7e8c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:12.868163 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc402aa-7330-4fbf-bbad-a4da5ab7e8c7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:12.868500 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc402aa-7330-4fbf-bbad-a4da5ab7e8c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:12.869877 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc402aa-7330-4fbf-bbad-a4da5ab7e8c7 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:12.890101 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 475/966] 10.222.0.15 () {56 vars in 1115 bytes} [Wed Aug 24 05:25:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:12.892366 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc402aa-7330-4fbf-bbad-a4da5ab7e8c7 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:12.940120 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:12.953842 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf57a9d-524e-4abf-9e59-36761faace7f None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:12.997200 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b1b035-6bd4-4108-a4d8-2a543a028cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:12.998421 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1b035-6bd4-4108-a4d8-2a543a028cfe None admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:12.998795 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1b035-6bd4-4108-a4d8-2a543a028cfe None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:12.999093 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1b035-6bd4-4108-a4d8-2a543a028cfe None admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:12.999902 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b1b035-6bd4-4108-a4d8-2a543a028cfe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:13.001558 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b1b035-6bd4-4108-a4d8-2a543a028cfe None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:13.025835 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc402aa-7330-4fbf-bbad-a4da5ab7e8c7 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:13.047439 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.sql.core [None req-e7b1b035-6bd4-4108-a4d8-2a543a028cfe None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 05:25:13.047439 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 05:25:13.047439 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [parameters: {'user_id': '4bed3ba6faf94324b48457b677f30e18', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 05:25:13.047439 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81670) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 05:25:13.059145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-e7b1b035-6bd4-4108-a4d8-2a543a028cfe None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 05:25:13.059145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:13.059145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 05:25:13.059145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 05:25:13.059145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 05:25:13.059145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 05:25:13.059145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 05:25:13.059145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 05:25:13.059145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 05:25:13.059145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application conn.query(q) Aug 24 05:25:13.059145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 05:25:13.059145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 05:25:13.059145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 05:25:13.059145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application result.read() Aug 24 05:25:13.059145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 05:25:13.060196 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 05:25:13.060196 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 05:25:13.060196 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 05:25:13.060196 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 05:25:13.060196 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 05:25:13.060196 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 05:25:13.060196 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 05:25:13.060196 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 05:25:13.060196 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:13.060196 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 05:25:13.060196 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:13.060196 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:13.060196 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 05:25:13.060196 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 05:25:13.060196 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 24 05:25:13.060196 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 24 05:25:13.060196 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 05:25:13.060196 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application next(self.gen) Aug 24 05:25:13.061177 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 05:25:13.061177 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application yield resource Aug 24 05:25:13.061177 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 05:25:13.061177 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application next(self.gen) Aug 24 05:25:13.061177 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 05:25:13.061177 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 05:25:13.061177 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 05:25:13.061177 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 05:25:13.061177 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 05:25:13.061177 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise self.value Aug 24 05:25:13.061177 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 05:25:13.061177 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 05:25:13.061177 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 05:25:13.061177 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application session.commit() Aug 24 05:25:13.061177 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 05:25:13.061177 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 05:25:13.061177 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 05:25:13.062063 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 05:25:13.062063 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 05:25:13.062063 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application self.session.flush() Aug 24 05:25:13.062063 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 05:25:13.062063 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 05:25:13.062063 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 05:25:13.062063 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 05:25:13.062063 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 05:25:13.062063 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application compat.raise_( Aug 24 05:25:13.062063 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 05:25:13.062063 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception Aug 24 05:25:13.062063 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 05:25:13.062063 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 05:25:13.062063 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 05:25:13.062063 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 05:25:13.062063 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 05:25:13.062063 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 05:25:13.062961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 05:25:13.062961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 05:25:13.062961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 24 05:25:13.062961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 24 05:25:13.062961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 05:25:13.062961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 05:25:13.062961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 05:25:13.062961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 05:25:13.062961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 05:25:13.062961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 05:25:13.062961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 05:25:13.062961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 05:25:13.062961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 05:25:13.062961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 05:25:13.062961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 05:25:13.062961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception Aug 24 05:25:13.063875 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 05:25:13.063875 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 05:25:13.063875 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 05:25:13.063875 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 05:25:13.063875 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 05:25:13.063875 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 05:25:13.063875 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 05:25:13.063875 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application conn.query(q) Aug 24 05:25:13.063875 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 05:25:13.063875 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 05:25:13.063875 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 05:25:13.063875 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application result.read() Aug 24 05:25:13.063875 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 05:25:13.063875 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 05:25:13.063875 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 05:25:13.063875 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 05:25:13.064751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 05:25:13.064751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 05:25:13.064751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 05:25:13.064751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 05:25:13.064751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 05:25:13.064751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 05:25:13.064751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [parameters: {'user_id': '4bed3ba6faf94324b48457b677f30e18', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 05:25:13.064751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 05:25:13.064751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:13.064751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:25:13.064751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:13.064751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:13.064751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:13.064751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:13.064751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:13.065546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:13.065546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:13.065546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:13.065546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:13.065546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:13.065546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:13.065546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:13.065546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 24 05:25:13.065546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 24 05:25:13.065546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:13.065546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:13.065546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:25:13.065546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:25:13.065546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:25:13.065546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:25:13.065546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 24 05:25:13.065546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 24 05:25:13.066482 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 24 05:25:13.066482 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 24 05:25:13.066482 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 05:25:13.066482 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 05:25:13.066482 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 05:25:13.066482 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:13.066482 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 492/967] 10.222.0.15 () {60 vars in 1131 bytes} [Wed Aug 24 05:25:12 2022] POST /identity/v3/users => generated 160 bytes in 239 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:25:13.068611 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b1b035-6bd4-4108-a4d8-2a543a028cfe None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:13.078843 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae3da5-4f5d-4c64-b82c-b4b9c986abc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:13.079415 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae3da5-4f5d-4c64-b82c-b4b9c986abc8 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:13.079687 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae3da5-4f5d-4c64-b82c-b4b9c986abc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:13.079963 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae3da5-4f5d-4c64-b82c-b4b9c986abc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:13.090995 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dae3da5-4f5d-4c64-b82c-b4b9c986abc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:13.093396 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dae3da5-4f5d-4c64-b82c-b4b9c986abc8 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:13.109765 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-2dae3da5-4f5d-4c64-b82c-b4b9c986abc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:25:13.109765 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:13.109765 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:13.109765 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:13.109765 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:13.109765 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:13.109765 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:13.109765 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:13.109765 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:13.109765 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:13.109765 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:13.109765 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:13.109765 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:25:13.109765 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:25:13.109765 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:25:13.109765 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:25:13.117933 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:13.117933 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:13.117933 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:13.117933 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:13.117933 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:13.117933 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:13.117933 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:13.117933 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:13.117933 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:13.117933 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:13.117933 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:13.117933 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:13.117933 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:13.117933 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:13.117933 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:13.117933 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:13.117933 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:25:13.118864 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:25:13.118864 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:25:13.118864 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:25:13.118864 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 476/968] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:25:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:25:13.118864 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae3da5-4f5d-4c64-b82c-b4b9c986abc8 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:13.130048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:13.202744 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b1b035-6bd4-4108-a4d8-2a543a028cfe None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:13.219235 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:13.232723 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae3da5-4f5d-4c64-b82c-b4b9c986abc8 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:13.241286 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa646592-31fa-4fa9-ac67-1e958be8b776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:13.241924 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa646592-31fa-4fa9-ac67-1e958be8b776 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:13.242208 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa646592-31fa-4fa9-ac67-1e958be8b776 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:13.242489 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa646592-31fa-4fa9-ac67-1e958be8b776 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:13.252513 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa646592-31fa-4fa9-ac67-1e958be8b776 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:13.254370 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa646592-31fa-4fa9-ac67-1e958be8b776 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:13.268879 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-aa646592-31fa-4fa9-ac67-1e958be8b776 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 05:25:13.268879 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:13.268879 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:13.268879 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:13.268879 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:13.268879 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:13.268879 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:13.268879 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:13.268879 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:13.268879 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:13.268879 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:13.268879 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:13.268879 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:25:13.268879 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:25:13.268879 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:25:13.268879 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:25:13.269908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:13.269908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:13.269908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:25:13.269908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:25:13.269908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:25:13.269908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:25:13.269908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:13.269908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:13.269908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:13.269908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:13.269908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:13.269908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:13.269908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:13.269908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:13.269908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:13.269908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:13.269908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:13.269908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:13.270918 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:13.270918 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:13.270918 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:25:13.270918 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:25:13.270918 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:25:13.270918 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:25:13.270918 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:25:13.270918 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:25:13.270918 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 05:25:13.270918 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:13.270918 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 493/969] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:25:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:25:13.272341 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d9687a-0380-49f7-ad38-784929707737 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:13.272341 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d9687a-0380-49f7-ad38-784929707737 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:13.272341 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d9687a-0380-49f7-ad38-784929707737 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:13.275993 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 494/970] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:25:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:25:13.280503 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d9687a-0380-49f7-ad38-784929707737 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:13.280662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e3967f-c183-4b28-94ac-da33a1e60ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:13.280662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3967f-c183-4b28-94ac-da33a1e60ca1 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:13.280846 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3967f-c183-4b28-94ac-da33a1e60ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:13.280989 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3967f-c183-4b28-94ac-da33a1e60ca1 None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:13.281906 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e3967f-c183-4b28-94ac-da33a1e60ca1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:13.282250 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e3967f-c183-4b28-94ac-da33a1e60ca1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:13.283626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e3967f-c183-4b28-94ac-da33a1e60ca1 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:13.301330 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 477/971] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:25:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:25:13.301330 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae3a959-82f1-4f5d-a298-a35bc6743786 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:13.301330 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae3a959-82f1-4f5d-a298-a35bc6743786 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:13.301330 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae3a959-82f1-4f5d-a298-a35bc6743786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:13.316646 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-8ae3a959-82f1-4f5d-a298-a35bc6743786 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:25:13.317848 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-8ae3a959-82f1-4f5d-a298-a35bc6743786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:25:13.390450 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 478/972] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:25:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:25:13.397092 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae3a959-82f1-4f5d-a298-a35bc6743786 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:13.398232 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d9687a-0380-49f7-ad38-784929707737 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:13.400070 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:13.439578 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2324bb-9db1-4e1b-8a0e-be878c352e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:13.439827 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2324bb-9db1-4e1b-8a0e-be878c352e43 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:13.439922 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2324bb-9db1-4e1b-8a0e-be878c352e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:13.440210 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2324bb-9db1-4e1b-8a0e-be878c352e43 None admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:13.441009 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2324bb-9db1-4e1b-8a0e-be878c352e43 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:13.441326 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2324bb-9db1-4e1b-8a0e-be878c352e43 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:13.442815 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2324bb-9db1-4e1b-8a0e-be878c352e43 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:13.488145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 495/973] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Aug 24 05:25:13 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:13.489488 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:13.514519 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae3a959-82f1-4f5d-a298-a35bc6743786 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:13.555113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf4a32-7a26-4d12-a5d1-c6713cce8e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:13.555763 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf4a32-7a26-4d12-a5d1-c6713cce8e03 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:13.556043 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf4a32-7a26-4d12-a5d1-c6713cce8e03 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:13.556326 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf4a32-7a26-4d12-a5d1-c6713cce8e03 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:13.565204 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acf4a32-7a26-4d12-a5d1-c6713cce8e03 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:13.566984 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acf4a32-7a26-4d12-a5d1-c6713cce8e03 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:13.584294 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-0acf4a32-7a26-4d12-a5d1-c6713cce8e03 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 05:25:13.584294 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:13.584294 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:13.584294 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:13.584294 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:13.584294 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:13.584294 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:13.584294 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:13.584294 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:13.584294 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:13.584294 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:13.584294 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:13.584294 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:25:13.584294 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:25:13.584294 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:25:13.584294 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:25:13.585350 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:13.585350 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:13.585350 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:25:13.585350 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:25:13.585350 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:25:13.585350 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:25:13.585350 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:13.585350 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:13.585350 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:13.585350 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:13.585350 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:13.585350 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:13.585350 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:13.585350 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:13.585350 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:13.585350 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:13.585350 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:13.585350 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:13.586293 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:13.586293 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:13.586293 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:25:13.586293 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:25:13.586293 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:25:13.586293 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:25:13.586293 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:25:13.586293 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:25:13.586293 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 05:25:13.586293 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:25:13.588874 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 479/974] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Aug 24 05:25:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:25:13.595960 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2324bb-9db1-4e1b-8a0e-be878c352e43 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:13.678909 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf4a32-7a26-4d12-a5d1-c6713cce8e03 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:13.713681 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2324bb-9db1-4e1b-8a0e-be878c352e43 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:13.761410 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4b500-0481-458e-89d6-f22049f71abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:13.762156 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4b500-0481-458e-89d6-f22049f71abd None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:13.762440 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4b500-0481-458e-89d6-f22049f71abd None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:13.762714 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4b500-0481-458e-89d6-f22049f71abd None admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:13.769450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec4b500-0481-458e-89d6-f22049f71abd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:13.769450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec4b500-0481-458e-89d6-f22049f71abd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:13.769450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec4b500-0481-458e-89d6-f22049f71abd None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:13.793421 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 496/975] 10.222.0.15 () {56 vars in 1149 bytes} [Wed Aug 24 05:25:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:13.795411 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4b500-0481-458e-89d6-f22049f71abd None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:13.803865 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf4a32-7a26-4d12-a5d1-c6713cce8e03 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:13.816576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:13.842261 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc25489-e1f1-42a3-af5b-12689a8a3959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:13.842805 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc25489-e1f1-42a3-af5b-12689a8a3959 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:13.843081 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc25489-e1f1-42a3-af5b-12689a8a3959 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:13.843353 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc25489-e1f1-42a3-af5b-12689a8a3959 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:13.843937 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc25489-e1f1-42a3-af5b-12689a8a3959 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:13.845320 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc25489-e1f1-42a3-af5b-12689a8a3959 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:13.854406 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-2bc25489-e1f1-42a3-af5b-12689a8a3959 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:25:13.854406 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:13.854406 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:25:13.854406 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:25:13.854406 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:25:13.854406 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:25:13.854406 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:25:13.854406 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:25:13.854406 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:25:13.854406 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:25:13.854406 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:25:13.854406 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:25:13.854406 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:13.854406 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:13.854406 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:13.854406 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:13.854406 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:13.855594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:13.855594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:13.855594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:13.855594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:13.855594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:13.855594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:13.855594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:25:13.855594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:25:13.855594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:25:13.855594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:25:13.855594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:13.855594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:13.855594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:13.855594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:13.855594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:13.855594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:13.856874 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:13.856874 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:13.856874 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:13.856874 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:13.856874 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:13.856874 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:13.856874 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:13.856874 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:13.856874 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:13.856874 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:13.856874 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:25:13.856874 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:25:13.856874 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:25:13.856874 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:25:13.856874 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 480/976] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Aug 24 05:25:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:25:13.861593 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc25489-e1f1-42a3-af5b-12689a8a3959 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:13.871409 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:13.909100 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4b500-0481-458e-89d6-f22049f71abd None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:13.945587 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92358ada-9922-46ea-a729-261c49f74e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:13.946120 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92358ada-9922-46ea-a729-261c49f74e2c None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:13.946408 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92358ada-9922-46ea-a729-261c49f74e2c None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:13.946680 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92358ada-9922-46ea-a729-261c49f74e2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:13.977283 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc25489-e1f1-42a3-af5b-12689a8a3959 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:14.014368 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccced576-308a-4f3d-a37c-bd49311870ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:14.014978 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccced576-308a-4f3d-a37c-bd49311870ba None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:14.015254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccced576-308a-4f3d-a37c-bd49311870ba None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:14.015536 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccced576-308a-4f3d-a37c-bd49311870ba None admin] PATH_INFO: `/v3/services` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:14.016168 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccced576-308a-4f3d-a37c-bd49311870ba None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:14.016498 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccced576-308a-4f3d-a37c-bd49311870ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:14.017882 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccced576-308a-4f3d-a37c-bd49311870ba None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:14.019869 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92358ada-9922-46ea-a729-261c49f74e2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:14.021379 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92358ada-9922-46ea-a729-261c49f74e2c None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:14.027902 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 481/977] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Aug 24 05:25:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:25:14.028877 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccced576-308a-4f3d-a37c-bd49311870ba None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:14.042391 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:14.104202 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:14.133257 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 497/978] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:25:13 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:25:14.138992 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92358ada-9922-46ea-a729-261c49f74e2c None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:14.150163 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccced576-308a-4f3d-a37c-bd49311870ba None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:14.193112 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b91481e-17b7-449e-a68f-ee558e084978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:14.193728 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b91481e-17b7-449e-a68f-ee558e084978 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:14.194011 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b91481e-17b7-449e-a68f-ee558e084978 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:14.194287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b91481e-17b7-449e-a68f-ee558e084978 None admin] PATH_INFO: `/v3/services` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:14.194921 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b91481e-17b7-449e-a68f-ee558e084978 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:14.195230 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b91481e-17b7-449e-a68f-ee558e084978 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:14.196672 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b91481e-17b7-449e-a68f-ee558e084978 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:14.202076 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 482/979] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Aug 24 05:25:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:14.207893 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b91481e-17b7-449e-a68f-ee558e084978 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:14.253113 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92358ada-9922-46ea-a729-261c49f74e2c None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:14.272122 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:14.291202 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a9c86-bb83-4e03-bd87-4836accc6858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:14.291917 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a9c86-bb83-4e03-bd87-4836accc6858 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:14.292189 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a9c86-bb83-4e03-bd87-4836accc6858 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:14.292500 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a9c86-bb83-4e03-bd87-4836accc6858 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:14.303394 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5a9c86-bb83-4e03-bd87-4836accc6858 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:14.305032 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5a9c86-bb83-4e03-bd87-4836accc6858 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:14.320237 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b91481e-17b7-449e-a68f-ee558e084978 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:14.320863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-be5a9c86-bb83-4e03-bd87-4836accc6858 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:25:14.320863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:14.320863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:14.320863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:14.320863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:14.320863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:14.320863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:14.320863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:14.320863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:14.320863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:14.320863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:14.320863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:14.320863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:25:14.320863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:25:14.320863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:25:14.320863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:25:14.321782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:14.321782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:14.321782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:14.321782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:14.321782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:14.321782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:14.321782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:14.321782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:14.321782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:14.321782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:14.321782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:14.321782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:14.321782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:14.321782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:14.321782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:14.321782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:14.321782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:25:14.322692 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:25:14.322692 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:25:14.322692 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:14.322692 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 498/980] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:25:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:25:14.328720 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a9c86-bb83-4e03-bd87-4836accc6858 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:14.353209 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:14.360092 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb3358b-851d-44cf-aa34-26ac562d942e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:14.360771 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3358b-851d-44cf-aa34-26ac562d942e None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:14.361082 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3358b-851d-44cf-aa34-26ac562d942e None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:14.361510 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3358b-851d-44cf-aa34-26ac562d942e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:14.362237 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb3358b-851d-44cf-aa34-26ac562d942e None admin] RBAC: Adding query filter params (interface=admin, service_id=584407fb63a0413fa870f82e2640c7b8, region_id=RegionOne) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:14.362555 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb3358b-851d-44cf-aa34-26ac562d942e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:14.363958 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb3358b-851d-44cf-aa34-26ac562d942e None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:14.369468 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 483/981] 10.222.0.15 () {56 vars in 1249 bytes} [Wed Aug 24 05:25:14 2022] GET /identity/v3/endpoints?service_id=584407fb63a0413fa870f82e2640c7b8&interface=admin®ion_id=RegionOne => generated 193 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:25:14.375861 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb3358b-851d-44cf-aa34-26ac562d942e None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:14.452730 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a9c86-bb83-4e03-bd87-4836accc6858 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:14.489768 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8385be-e631-4bc7-a47c-3161a6e19425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:14.490372 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8385be-e631-4bc7-a47c-3161a6e19425 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:14.490656 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8385be-e631-4bc7-a47c-3161a6e19425 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:14.490939 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8385be-e631-4bc7-a47c-3161a6e19425 None admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:14.491620 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8385be-e631-4bc7-a47c-3161a6e19425 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:14.491926 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8385be-e631-4bc7-a47c-3161a6e19425 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:14.493408 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8385be-e631-4bc7-a47c-3161a6e19425 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:14.495659 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb3358b-851d-44cf-aa34-26ac562d942e None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:14.505513 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 499/982] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:25:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:14.506866 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6ead9-3464-48bd-b3ae-b276ccdee647 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:14.507163 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6ead9-3464-48bd-b3ae-b276ccdee647 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:14.507442 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6ead9-3464-48bd-b3ae-b276ccdee647 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:14.508717 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 500/983] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:25:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:25:14.514796 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0670c-e455-4101-a65b-fed70eec260c None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:14.515085 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0670c-e455-4101-a65b-fed70eec260c None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:14.515450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0670c-e455-4101-a65b-fed70eec260c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:14.535020 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-e7f0670c-e455-4101-a65b-fed70eec260c None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:25:14.536590 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-e7f0670c-e455-4101-a65b-fed70eec260c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:25:14.537411 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4965d3-1516-466a-8fa5-5ee1a23aa2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:14.538091 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4965d3-1516-466a-8fa5-5ee1a23aa2f8 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:14.538377 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4965d3-1516-466a-8fa5-5ee1a23aa2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:14.538663 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4965d3-1516-466a-8fa5-5ee1a23aa2f8 None admin] PATH_INFO: `/v3/services` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:14.539308 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4965d3-1516-466a-8fa5-5ee1a23aa2f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:14.540715 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4965d3-1516-466a-8fa5-5ee1a23aa2f8 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:14.547864 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 484/984] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Aug 24 05:25:14 2022] GET /identity/v3/services => generated 1497 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:25:14.548732 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4965d3-1516-466a-8fa5-5ee1a23aa2f8 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:14.563222 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:14.627552 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 501/985] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:25:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:25:14.630347 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:14.683913 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4965d3-1516-466a-8fa5-5ee1a23aa2f8 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:14.722458 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d48fb1-facd-4980-a37c-a84e8464a254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:14.723025 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d48fb1-facd-4980-a37c-a84e8464a254 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:14.723311 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d48fb1-facd-4980-a37c-a84e8464a254 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:14.723584 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d48fb1-facd-4980-a37c-a84e8464a254 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:14.730986 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d48fb1-facd-4980-a37c-a84e8464a254 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:14.733048 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d48fb1-facd-4980-a37c-a84e8464a254 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:14.744709 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-30d48fb1-facd-4980-a37c-a84e8464a254 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:25:14.744709 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:14.744709 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:14.744709 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:14.744709 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:14.744709 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:14.744709 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:14.744709 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:14.744709 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:14.744709 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:14.744709 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:14.744709 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:14.744709 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:25:14.744709 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:25:14.744709 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:25:14.745626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:25:14.745626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:14.745626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:14.745626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:14.745626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:14.745626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:14.745626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:14.745626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:14.745626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:14.745626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:14.745626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:14.745626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:14.745626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:14.745626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:14.745626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:14.745626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:14.745626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:14.746575 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:25:14.746575 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:25:14.746575 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:25:14.746575 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:25:14.746575 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:25:14.746575 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:25:14.746575 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:25:14.746575 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:25:14.746575 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 485/986] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:25:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:25:14.752340 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f0670c-e455-4101-a65b-fed70eec260c None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:14.852668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f0670c-e455-4101-a65b-fed70eec260c None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:14.888909 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeef72f-3b6f-4773-a472-afb8b79c1be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:14.889556 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeef72f-3b6f-4773-a472-afb8b79c1be0 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:14.889836 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeef72f-3b6f-4773-a472-afb8b79c1be0 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:14.890109 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeef72f-3b6f-4773-a472-afb8b79c1be0 None admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:14.890811 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeef72f-3b6f-4773-a472-afb8b79c1be0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:14.891121 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeef72f-3b6f-4773-a472-afb8b79c1be0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:14.892594 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeef72f-3b6f-4773-a472-afb8b79c1be0 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:14.892700 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d48fb1-facd-4980-a37c-a84e8464a254 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:14.905862 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 502/987] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Aug 24 05:25:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:14.917615 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeef72f-3b6f-4773-a472-afb8b79c1be0 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:14.942164 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:15.007077 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d48fb1-facd-4980-a37c-a84e8464a254 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:15.024776 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:15.027568 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeef72f-3b6f-4773-a472-afb8b79c1be0 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:15.042752 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e44339-a934-44ca-adc6-2f23f50596cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:15.043377 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e44339-a934-44ca-adc6-2f23f50596cd None admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:15.043640 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e44339-a934-44ca-adc6-2f23f50596cd None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:15.043916 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e44339-a934-44ca-adc6-2f23f50596cd None admin] PATH_INFO: `/v3/services` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:15.044643 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e44339-a934-44ca-adc6-2f23f50596cd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:15.046005 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e44339-a934-44ca-adc6-2f23f50596cd None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:15.065172 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0703ec1a-2b38-4192-9af3-20db4406cd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:15.065890 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703ec1a-2b38-4192-9af3-20db4406cd45 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:15.066232 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703ec1a-2b38-4192-9af3-20db4406cd45 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:15.066571 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703ec1a-2b38-4192-9af3-20db4406cd45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:15.067357 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0703ec1a-2b38-4192-9af3-20db4406cd45 None admin] RBAC: Adding query filter params (role.id=719a48d7876b4ffa9489dcc5d87cfbad, scope.project.id=59592706ae9442e08353158c625e9581, user.id=604380c191c8461bbb937b8d951238bc) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:15.067729 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0703ec1a-2b38-4192-9af3-20db4406cd45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:15.069227 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0703ec1a-2b38-4192-9af3-20db4406cd45 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:15.074971 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 503/988] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:25:14 2022] GET /identity/v3/role_assignments?user.id=604380c191c8461bbb937b8d951238bc&scope.project.id=59592706ae9442e08353158c625e9581&role.id=719a48d7876b4ffa9489dcc5d87cfbad => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:25:15.124441 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 486/989] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Aug 24 05:25:14 2022] POST /identity/v3/services => generated 242 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:25:16.501421 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe726b0a-d1a9-453f-88ac-d03e9888f74b None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:16.501914 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe726b0a-d1a9-453f-88ac-d03e9888f74b None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:16.501914 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe726b0a-d1a9-453f-88ac-d03e9888f74b None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:16.503353 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 504/990] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:25:16.508426 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a3826-d075-4712-845b-f385d9ebd1fc None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:16.508934 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a3826-d075-4712-845b-f385d9ebd1fc None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:16.509012 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a3826-d075-4712-845b-f385d9ebd1fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:16.527555 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-1c0a3826-d075-4712-845b-f385d9ebd1fc None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:25:16.529300 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-1c0a3826-d075-4712-845b-f385d9ebd1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:25:16.613270 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 487/991] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:25:16 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:25:16.878950 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe726b0a-d1a9-453f-88ac-d03e9888f74b None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:16.975896 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:16.993478 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe726b0a-d1a9-453f-88ac-d03e9888f74b None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:17.029276 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72399502-11de-49ac-9dc0-1fcb57760cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:17.029900 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72399502-11de-49ac-9dc0-1fcb57760cb6 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:17.030178 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72399502-11de-49ac-9dc0-1fcb57760cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:17.030448 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72399502-11de-49ac-9dc0-1fcb57760cb6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:17.031025 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72399502-11de-49ac-9dc0-1fcb57760cb6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:17.032692 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72399502-11de-49ac-9dc0-1fcb57760cb6 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:17.041651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-72399502-11de-49ac-9dc0-1fcb57760cb6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:25:17.041651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:17.041651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:25:17.041651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:25:17.041651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:25:17.041651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:25:17.041651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:25:17.041651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:25:17.041651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:25:17.041651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:17.041651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:25:17.041651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:17.041651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:17.041651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:17.041651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:17.041651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:17.041651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:17.042647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:17.042647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:17.042647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:17.042647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:17.042647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:17.042647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:17.042647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:25:17.042647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:25:17.042647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:25:17.042647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:25:17.042647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:17.042647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:17.042647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:17.042647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:17.042647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:17.042647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:17.043507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:17.043507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:17.043507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:17.043507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:17.043507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:17.043507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:17.043507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:17.043507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:17.043507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:17.043507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:17.043507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:25:17.043507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:25:17.043507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:25:17.043507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:17.046189 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67760d53-6171-4d6f-9d1a-2253bdbc9eb1 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:17.046475 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67760d53-6171-4d6f-9d1a-2253bdbc9eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:17.046751 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67760d53-6171-4d6f-9d1a-2253bdbc9eb1 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:17.047697 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 505/992] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Aug 24 05:25:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:25:17.048240 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 488/993] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:25:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:25:17.048573 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72399502-11de-49ac-9dc0-1fcb57760cb6 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:17.053535 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f512d97-3ca4-4579-b35d-7fed0be18293 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:17.053936 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f512d97-3ca4-4579-b35d-7fed0be18293 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:17.054277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f512d97-3ca4-4579-b35d-7fed0be18293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:17.071914 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-9f512d97-3ca4-4579-b35d-7fed0be18293 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:25:17.073244 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-9f512d97-3ca4-4579-b35d-7fed0be18293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:25:17.143925 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 489/994] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:25:17 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:25:17.157386 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72399502-11de-49ac-9dc0-1fcb57760cb6 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:17.186827 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db8caf-6f65-4aa9-87b8-23ed35d3548c None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:17.187112 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db8caf-6f65-4aa9-87b8-23ed35d3548c None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:17.187388 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db8caf-6f65-4aa9-87b8-23ed35d3548c None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:17.188873 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 490/995] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:25:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:25:17.193813 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86a3ee-fac9-4fe5-87ec-a94327dc0f34 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:17.194099 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86a3ee-fac9-4fe5-87ec-a94327dc0f34 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:17.194377 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86a3ee-fac9-4fe5-87ec-a94327dc0f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:17.203522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae41bb1-79e0-4975-ba29-45c142cd1ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:17.204139 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae41bb1-79e0-4975-ba29-45c142cd1ffe None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:17.204445 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae41bb1-79e0-4975-ba29-45c142cd1ffe None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:17.204731 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae41bb1-79e0-4975-ba29-45c142cd1ffe None admin] PATH_INFO: `/v3/services` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:17.205369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae41bb1-79e0-4975-ba29-45c142cd1ffe None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:17.205677 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae41bb1-79e0-4975-ba29-45c142cd1ffe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:17.207049 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae41bb1-79e0-4975-ba29-45c142cd1ffe None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:17.216671 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 506/996] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Aug 24 05:25:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:17.216879 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-5a86a3ee-fac9-4fe5-87ec-a94327dc0f34 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:25:17.216879 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-5a86a3ee-fac9-4fe5-87ec-a94327dc0f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:25:17.220906 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae41bb1-79e0-4975-ba29-45c142cd1ffe None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:17.240070 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:17.245032 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:17.290419 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 491/997] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:25:17 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:25:17.341374 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae41bb1-79e0-4975-ba29-45c142cd1ffe None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:17.376914 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a19fba-2b4a-445e-b8a7-7aadbb9f61a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:17.377522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a19fba-2b4a-445e-b8a7-7aadbb9f61a1 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:17.377802 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a19fba-2b4a-445e-b8a7-7aadbb9f61a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:17.378075 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a19fba-2b4a-445e-b8a7-7aadbb9f61a1 None admin] PATH_INFO: `/v3/services` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:17.378700 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a19fba-2b4a-445e-b8a7-7aadbb9f61a1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:17.379002 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a19fba-2b4a-445e-b8a7-7aadbb9f61a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:17.380390 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a19fba-2b4a-445e-b8a7-7aadbb9f61a1 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:17.385418 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 507/998] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Aug 24 05:25:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:25:17.392024 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a86a3ee-fac9-4fe5-87ec-a94327dc0f34 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:17.433059 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a19fba-2b4a-445e-b8a7-7aadbb9f61a1 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:17.500169 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a86a3ee-fac9-4fe5-87ec-a94327dc0f34 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:17.520824 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:17.538416 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6114e4-1c45-4453-9755-9d560da69af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:17.539043 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6114e4-1c45-4453-9755-9d560da69af0 None admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:17.539302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6114e4-1c45-4453-9755-9d560da69af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:17.539581 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6114e4-1c45-4453-9755-9d560da69af0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:17.540253 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6114e4-1c45-4453-9755-9d560da69af0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:17.541647 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6114e4-1c45-4453-9755-9d560da69af0 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:17.546227 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a19fba-2b4a-445e-b8a7-7aadbb9f61a1 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:17.581843 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3123b7-6103-4918-811d-3c77b44d1179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:17.582522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3123b7-6103-4918-811d-3c77b44d1179 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:17.582804 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3123b7-6103-4918-811d-3c77b44d1179 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:17.583079 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3123b7-6103-4918-811d-3c77b44d1179 None admin] PATH_INFO: `/v3/services/image` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:17.583662 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3123b7-6103-4918-811d-3c77b44d1179 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:17.585061 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3123b7-6103-4918-811d-3c77b44d1179 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:17.594576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-da3123b7-6103-4918-811d-3c77b44d1179 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 05:25:17.594576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:17.594576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:25:17.594576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:25:17.594576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:25:17.594576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:25:17.594576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:25:17.594576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:25:17.594576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 05:25:17.594576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:17.594576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:25:17.594576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:17.594576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:17.594576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:17.594576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:17.594576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:17.594576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:17.601620 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:17.601620 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:17.601620 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:17.601620 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:17.601620 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:17.601620 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:17.601620 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:25:17.601620 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:25:17.601620 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:25:17.601620 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:25:17.601620 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:17.601620 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:17.601620 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:17.601620 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:17.601620 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:17.601620 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:17.602493 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:17.602493 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:17.602493 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:17.602493 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:17.602493 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:17.602493 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:17.602493 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:17.602493 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:17.602493 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:17.602493 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:17.602493 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:25:17.602493 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:25:17.602493 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 05:25:17.602493 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:17.602493 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 508/999] 10.222.0.15 () {56 vars in 1105 bytes} [Wed Aug 24 05:25:17 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:25:17.603398 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3123b7-6103-4918-811d-3c77b44d1179 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:17.635972 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 492/1000] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:25:17 2022] POST /identity/v3/endpoints => generated 325 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:25:17.636870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6114e4-1c45-4453-9755-9d560da69af0 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:17.666161 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:17.717862 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3123b7-6103-4918-811d-3c77b44d1179 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:17.754153 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6114e4-1c45-4453-9755-9d560da69af0 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:17.758108 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90971e-134d-4f58-876f-ee549b0e2950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:17.758756 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90971e-134d-4f58-876f-ee549b0e2950 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:17.759041 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90971e-134d-4f58-876f-ee549b0e2950 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:17.759315 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90971e-134d-4f58-876f-ee549b0e2950 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:17.759895 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df90971e-134d-4f58-876f-ee549b0e2950 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:17.761320 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df90971e-134d-4f58-876f-ee549b0e2950 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:17.772090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-df90971e-134d-4f58-876f-ee549b0e2950 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 05:25:17.772090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:17.772090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:25:17.772090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:25:17.772090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:25:17.772090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:25:17.772090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:25:17.772090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:25:17.772090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 05:25:17.772090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:17.772090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:25:17.772090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:17.772090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:17.772090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:17.772090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:17.772090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:17.772090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:17.773462 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:17.773462 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:17.773462 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:17.773462 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:17.773462 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:17.773462 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:17.773462 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:25:17.773462 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:25:17.773462 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:25:17.773462 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:25:17.773462 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:17.773462 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:17.773462 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:17.773462 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:17.773462 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:17.773462 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:17.774518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:17.774518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:17.774518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:17.774518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:17.774518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:17.774518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:17.774518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:17.774518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:17.774518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:17.774518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:17.774518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:25:17.774518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:25:17.774518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 05:25:17.774518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:17.778398 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 509/1001] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:25:17 2022] GET /identity/v3/services/compute => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:25:17.779900 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90971e-134d-4f58-876f-ee549b0e2950 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:17.784836 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:17.806512 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f4d00-3a5a-469a-a651-d5b1b858d36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:17.807309 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f4d00-3a5a-469a-a651-d5b1b858d36c None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:17.808245 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f4d00-3a5a-469a-a651-d5b1b858d36c None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:17.808566 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f4d00-3a5a-469a-a651-d5b1b858d36c None admin] PATH_INFO: `/v3/services` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:17.809203 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289f4d00-3a5a-469a-a651-d5b1b858d36c None admin] RBAC: Adding query filter params (name=image) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:17.809515 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289f4d00-3a5a-469a-a651-d5b1b858d36c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:17.811633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289f4d00-3a5a-469a-a651-d5b1b858d36c None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:17.817533 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 493/1002] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:25:17 2022] GET /identity/v3/services?name=image => generated 122 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:17.823215 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f4d00-3a5a-469a-a651-d5b1b858d36c None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:17.893507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90971e-134d-4f58-876f-ee549b0e2950 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:17.917546 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:17.934269 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f5ad2e-68b0-4c31-a7c8-8acf55c6ed56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:17.935432 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f5ad2e-68b0-4c31-a7c8-8acf55c6ed56 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:17.935712 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f5ad2e-68b0-4c31-a7c8-8acf55c6ed56 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:17.935987 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f5ad2e-68b0-4c31-a7c8-8acf55c6ed56 None admin] PATH_INFO: `/v3/services` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:17.936087 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f4d00-3a5a-469a-a651-d5b1b858d36c None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:17.936652 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f5ad2e-68b0-4c31-a7c8-8acf55c6ed56 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:17.936990 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f5ad2e-68b0-4c31-a7c8-8acf55c6ed56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:17.938362 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f5ad2e-68b0-4c31-a7c8-8acf55c6ed56 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:17.948731 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 510/1003] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Aug 24 05:25:17 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:17.949896 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f5ad2e-68b0-4c31-a7c8-8acf55c6ed56 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:17.978552 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d7525-eb06-433e-b4f8-fb5b99734818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:17.979406 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d7525-eb06-433e-b4f8-fb5b99734818 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:17.979745 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d7525-eb06-433e-b4f8-fb5b99734818 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:17.979966 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d7525-eb06-433e-b4f8-fb5b99734818 None admin] PATH_INFO: `/v3/services` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:17.980634 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986d7525-eb06-433e-b4f8-fb5b99734818 None admin] RBAC: Adding query filter params (type=image) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:17.981132 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986d7525-eb06-433e-b4f8-fb5b99734818 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:17.982971 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986d7525-eb06-433e-b4f8-fb5b99734818 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:17.987809 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 494/1004] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:25:17 2022] GET /identity/v3/services?type=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:25:18.039033 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:18.077447 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f5ad2e-68b0-4c31-a7c8-8acf55c6ed56 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:18.115204 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8151cc23-184a-4208-ada7-63a26607f086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:18.115834 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151cc23-184a-4208-ada7-63a26607f086 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:18.116119 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151cc23-184a-4208-ada7-63a26607f086 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:18.116392 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151cc23-184a-4208-ada7-63a26607f086 None admin] PATH_INFO: `/v3/services` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:18.117063 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8151cc23-184a-4208-ada7-63a26607f086 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:18.117367 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8151cc23-184a-4208-ada7-63a26607f086 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:18.118774 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8151cc23-184a-4208-ada7-63a26607f086 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:18.125403 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 511/1005] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Aug 24 05:25:17 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:18.131161 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d7525-eb06-433e-b4f8-fb5b99734818 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:18.245099 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d7525-eb06-433e-b4f8-fb5b99734818 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:18.286744 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f22697-2a3e-41a3-bf0c-3fa74c69e38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:18.287362 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f22697-2a3e-41a3-bf0c-3fa74c69e38d None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:18.287642 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f22697-2a3e-41a3-bf0c-3fa74c69e38d None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:18.287915 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f22697-2a3e-41a3-bf0c-3fa74c69e38d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:18.288678 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f22697-2a3e-41a3-bf0c-3fa74c69e38d None admin] RBAC: Adding query filter params (interface=public, service_id=2cc30c0a6b924f6facb9a977eae238c4, region_id=RegionOne) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:18.288991 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f22697-2a3e-41a3-bf0c-3fa74c69e38d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:18.290367 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f22697-2a3e-41a3-bf0c-3fa74c69e38d None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:18.296934 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 495/1006] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Aug 24 05:25:18 2022] GET /identity/v3/endpoints?service_id=2cc30c0a6b924f6facb9a977eae238c4&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:18.302388 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8151cc23-184a-4208-ada7-63a26607f086 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:18.413765 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8151cc23-184a-4208-ada7-63a26607f086 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:18.444009 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:18.452494 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419540b3-af27-483f-a368-c6db6f424773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:18.453117 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419540b3-af27-483f-a368-c6db6f424773 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:18.453402 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419540b3-af27-483f-a368-c6db6f424773 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:18.453682 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419540b3-af27-483f-a368-c6db6f424773 None admin] PATH_INFO: `/v3/services` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:18.454294 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419540b3-af27-483f-a368-c6db6f424773 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:18.455886 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419540b3-af27-483f-a368-c6db6f424773 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:18.464093 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 512/1007] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Aug 24 05:25:18 2022] GET /identity/v3/services => generated 1727 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:25:19.648169 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc3d7d-5266-4d9a-bec4-6b692c756abb None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:19.648715 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc3d7d-5266-4d9a-bec4-6b692c756abb None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:19.648795 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc3d7d-5266-4d9a-bec4-6b692c756abb None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:19.650314 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 496/1008] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:25:19.656770 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd27c1-2703-46ce-a01d-94ca38794e56 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:19.657187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd27c1-2703-46ce-a01d-94ca38794e56 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:19.657350 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd27c1-2703-46ce-a01d-94ca38794e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:19.675505 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-addd27c1-2703-46ce-a01d-94ca38794e56 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:25:19.676731 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-addd27c1-2703-46ce-a01d-94ca38794e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:25:19.743304 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 513/1009] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:25:19 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:25:19.936297 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb75765-52c3-447b-a520-ab3e84aaa6b9 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:19.936681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb75765-52c3-447b-a520-ab3e84aaa6b9 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:19.936792 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb75765-52c3-447b-a520-ab3e84aaa6b9 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:19.940459 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 497/1010] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:25:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:25:19.943471 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ec9e5-0e80-4c4f-8e6c-f024375f0531 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:19.943766 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ec9e5-0e80-4c4f-8e6c-f024375f0531 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:19.944044 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ec9e5-0e80-4c4f-8e6c-f024375f0531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:19.962651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-127ec9e5-0e80-4c4f-8e6c-f024375f0531 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:25:19.963853 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-127ec9e5-0e80-4c4f-8e6c-f024375f0531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:25:20.039613 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb75765-52c3-447b-a520-ab3e84aaa6b9 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:20.044465 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 514/1011] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:25:19 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:25:20.057145 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:20.146099 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb75765-52c3-447b-a520-ab3e84aaa6b9 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:20.188339 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f540e-32bc-4ad7-92f3-7d482175becb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:20.189044 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f540e-32bc-4ad7-92f3-7d482175becb None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:20.189329 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f540e-32bc-4ad7-92f3-7d482175becb None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:20.189605 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f540e-32bc-4ad7-92f3-7d482175becb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:20.190773 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76f540e-32bc-4ad7-92f3-7d482175becb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:20.192274 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76f540e-32bc-4ad7-92f3-7d482175becb None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:20.196440 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 498/1012] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:25:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:25:20.199755 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ec9e5-0e80-4c4f-8e6c-f024375f0531 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:20.258792 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:20.303502 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ec9e5-0e80-4c4f-8e6c-f024375f0531 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:20.317068 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f540e-32bc-4ad7-92f3-7d482175becb None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:20.340514 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cbb12d-299d-40f6-ab73-177473dcc8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:20.341915 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cbb12d-299d-40f6-ab73-177473dcc8c8 None admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:20.342452 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cbb12d-299d-40f6-ab73-177473dcc8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:20.342735 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cbb12d-299d-40f6-ab73-177473dcc8c8 None admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:20.343400 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cbb12d-299d-40f6-ab73-177473dcc8c8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:20.344927 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cbb12d-299d-40f6-ab73-177473dcc8c8 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:20.407020 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:20.426011 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f540e-32bc-4ad7-92f3-7d482175becb None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:20.459257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f64a1a-56ff-4317-803d-0dc46ab6e1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:20.459785 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f64a1a-56ff-4317-803d-0dc46ab6e1ed None admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:20.460067 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f64a1a-56ff-4317-803d-0dc46ab6e1ed None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:20.460340 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f64a1a-56ff-4317-803d-0dc46ab6e1ed None admin] PATH_INFO: `/v3/services` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:20.461017 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f64a1a-56ff-4317-803d-0dc46ab6e1ed None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:20.462446 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f64a1a-56ff-4317-803d-0dc46ab6e1ed None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:20.492147 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 515/1013] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Aug 24 05:25:20 2022] POST /identity/v3/projects => generated 311 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:25:20.498167 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca352786-1af0-48fe-964c-6f836a5dc174 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:20.498451 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca352786-1af0-48fe-964c-6f836a5dc174 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:20.498730 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca352786-1af0-48fe-964c-6f836a5dc174 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:20.500157 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 516/1014] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:25:20.506361 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578870b-3a72-456d-962b-489aa1272b3a None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:20.506799 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578870b-3a72-456d-962b-489aa1272b3a None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:20.507017 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578870b-3a72-456d-962b-489aa1272b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:20.527008 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-4578870b-3a72-456d-962b-489aa1272b3a None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:25:20.528948 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-4578870b-3a72-456d-962b-489aa1272b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:25:20.600345 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 499/1015] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Aug 24 05:25:20 2022] POST /identity/v3/services => generated 242 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:25:20.614937 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 517/1016] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:25:20 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:25:20.891528 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f64a1a-56ff-4317-803d-0dc46ab6e1ed None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:21.002043 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f64a1a-56ff-4317-803d-0dc46ab6e1ed None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:21.037950 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9870b09-6821-4e34-aee7-f549a1961b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:21.038587 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9870b09-6821-4e34-aee7-f549a1961b09 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:21.038868 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9870b09-6821-4e34-aee7-f549a1961b09 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:21.039138 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9870b09-6821-4e34-aee7-f549a1961b09 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:21.039721 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9870b09-6821-4e34-aee7-f549a1961b09 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:21.041107 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9870b09-6821-4e34-aee7-f549a1961b09 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:21.050103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-b9870b09-6821-4e34-aee7-f549a1961b09 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 05:25:21.050103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:21.050103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:25:21.050103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:25:21.050103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:25:21.050103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:25:21.050103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:25:21.050103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:25:21.050103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 05:25:21.050103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:25:21.050103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:25:21.050103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:25:21.050103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:21.050103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:21.050103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:21.050103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:21.050103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:21.051113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:21.051113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:21.051113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:21.051113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:21.051113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:21.051113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:21.051113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:25:21.051113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:25:21.051113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:25:21.051113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:25:21.051113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:21.051113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:21.051113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:21.051113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:21.051113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:21.051113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:21.051961 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:21.051961 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:21.051961 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:21.051961 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:21.051961 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:21.051961 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:21.051961 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:21.051961 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:21.051961 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:21.051961 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:21.051961 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:25:21.051961 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:25:21.051961 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 05:25:21.051961 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:25:21.051961 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 500/1017] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:25:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:25:21.058337 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4578870b-3a72-456d-962b-489aa1272b3a None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:21.095375 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:21.166986 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4578870b-3a72-456d-962b-489aa1272b3a None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:21.207656 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e538ca3a-0e6d-4c41-9fed-0f2912cd7a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:21.208275 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e538ca3a-0e6d-4c41-9fed-0f2912cd7a70 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:21.208601 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e538ca3a-0e6d-4c41-9fed-0f2912cd7a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:21.208882 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e538ca3a-0e6d-4c41-9fed-0f2912cd7a70 None admin] PATH_INFO: `/v3/services` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:21.209516 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e538ca3a-0e6d-4c41-9fed-0f2912cd7a70 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:21.209822 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e538ca3a-0e6d-4c41-9fed-0f2912cd7a70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:21.211184 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e538ca3a-0e6d-4c41-9fed-0f2912cd7a70 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:21.221675 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 518/1018] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Aug 24 05:25:21 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:21.222655 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9870b09-6821-4e34-aee7-f549a1961b09 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:21.227029 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:21.329363 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9870b09-6821-4e34-aee7-f549a1961b09 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:21.370206 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765af882-36fb-462a-b251-4b8ea98034ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:21.370843 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765af882-36fb-462a-b251-4b8ea98034ba None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:21.371127 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765af882-36fb-462a-b251-4b8ea98034ba None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:21.371403 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765af882-36fb-462a-b251-4b8ea98034ba None admin] PATH_INFO: `/v3/services` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:21.372439 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765af882-36fb-462a-b251-4b8ea98034ba None admin] RBAC: Adding query filter params (type=compute) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:21.372917 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765af882-36fb-462a-b251-4b8ea98034ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:21.374291 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765af882-36fb-462a-b251-4b8ea98034ba None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:21.379916 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 501/1019] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Aug 24 05:25:21 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:21.385460 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e538ca3a-0e6d-4c41-9fed-0f2912cd7a70 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:21.486025 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:21.490438 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e538ca3a-0e6d-4c41-9fed-0f2912cd7a70 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:21.526750 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62750dc-c0a0-401a-8901-e54d5224ac6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:21.527368 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62750dc-c0a0-401a-8901-e54d5224ac6c None admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:21.527652 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62750dc-c0a0-401a-8901-e54d5224ac6c None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:21.527932 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62750dc-c0a0-401a-8901-e54d5224ac6c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:21.529051 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62750dc-c0a0-401a-8901-e54d5224ac6c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:21.530419 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62750dc-c0a0-401a-8901-e54d5224ac6c None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:21.625623 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 519/1020] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:25:21 2022] POST /identity/v3/endpoints => generated 334 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:25:22.448250 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c9c41-67d9-4fa1-afdc-1bf3aa7a9c7c None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:22.448795 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c9c41-67d9-4fa1-afdc-1bf3aa7a9c7c None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:22.448795 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c9c41-67d9-4fa1-afdc-1bf3aa7a9c7c None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:22.450838 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 502/1021] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:25:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:25:22.456064 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d296753-9319-48f6-a1ef-55fe1fb32a65 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:22.456498 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d296753-9319-48f6-a1ef-55fe1fb32a65 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:22.456710 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d296753-9319-48f6-a1ef-55fe1fb32a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:22.475686 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-2d296753-9319-48f6-a1ef-55fe1fb32a65 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:25:22.476929 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-2d296753-9319-48f6-a1ef-55fe1fb32a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:25:22.555800 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 520/1022] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:25:22 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:25:22.556235 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1063f39f-8ee4-4b5d-a71a-257f3f0b2cea None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:22.556561 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1063f39f-8ee4-4b5d-a71a-257f3f0b2cea None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:22.556870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1063f39f-8ee4-4b5d-a71a-257f3f0b2cea None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:22.558268 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 503/1023] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:25:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:25:22.563821 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3f5d1-f840-4c4c-b491-0ddfb73b8376 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:22.564115 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3f5d1-f840-4c4c-b491-0ddfb73b8376 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:22.564411 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3f5d1-f840-4c4c-b491-0ddfb73b8376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:22.582941 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-bba3f5d1-f840-4c4c-b491-0ddfb73b8376 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:25:22.584135 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-bba3f5d1-f840-4c4c-b491-0ddfb73b8376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:25:22.655443 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 521/1024] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:25:22 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:25:22.836290 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1063f39f-8ee4-4b5d-a71a-257f3f0b2cea None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:22.909277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:22.918171 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3f5d1-f840-4c4c-b491-0ddfb73b8376 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:22.930089 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:22.945220 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1063f39f-8ee4-4b5d-a71a-257f3f0b2cea None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:22.986647 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec228724-ad42-423a-88be-d5822c911751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:22.987284 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec228724-ad42-423a-88be-d5822c911751 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:22.987562 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec228724-ad42-423a-88be-d5822c911751 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:22.987849 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec228724-ad42-423a-88be-d5822c911751 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:22.989047 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec228724-ad42-423a-88be-d5822c911751 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:22.990479 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec228724-ad42-423a-88be-d5822c911751 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:22.992080 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 504/1025] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:25:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:22.997912 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec228724-ad42-423a-88be-d5822c911751 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:23.036492 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3f5d1-f840-4c4c-b491-0ddfb73b8376 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:23.071671 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004d2489-ba06-40d3-8df7-0c3efef730c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:23.072329 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d2489-ba06-40d3-8df7-0c3efef730c3 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:23.072645 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d2489-ba06-40d3-8df7-0c3efef730c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:23.072920 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d2489-ba06-40d3-8df7-0c3efef730c3 None admin] PATH_INFO: `/v3/services/image` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:23.073495 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004d2489-ba06-40d3-8df7-0c3efef730c3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:23.074864 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004d2489-ba06-40d3-8df7-0c3efef730c3 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:23.084304 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-004d2489-ba06-40d3-8df7-0c3efef730c3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 05:25:23.084304 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:23.084304 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:25:23.084304 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:25:23.084304 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:25:23.084304 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:25:23.084304 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:25:23.084304 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:25:23.084304 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 05:25:23.084304 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:23.084304 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:25:23.084304 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:23.084304 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:23.084304 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:23.084304 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:23.084304 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:23.084304 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:23.085618 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:23.085618 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:23.085618 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:23.085618 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:23.085618 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:23.085618 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:23.085618 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:25:23.085618 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:25:23.085618 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:25:23.085618 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:25:23.085618 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:23.085618 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:23.085618 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:23.085618 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:23.085618 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:23.085618 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:23.086892 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:23.086892 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:23.086892 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:23.086892 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:23.086892 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:23.086892 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:23.086892 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:23.086892 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:23.086892 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:23.086892 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:23.086892 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:25:23.086892 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:25:23.086892 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 05:25:23.086892 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:23.086892 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 522/1026] 10.222.0.15 () {56 vars in 1105 bytes} [Wed Aug 24 05:25:22 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:25:23.091457 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004d2489-ba06-40d3-8df7-0c3efef730c3 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:23.109746 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec228724-ad42-423a-88be-d5822c911751 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:23.147192 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad655428-de9e-4492-b4f9-1ed53b09b4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:23.147810 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad655428-de9e-4492-b4f9-1ed53b09b4dd None admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:23.148093 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad655428-de9e-4492-b4f9-1ed53b09b4dd None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:23.148373 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad655428-de9e-4492-b4f9-1ed53b09b4dd None admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:23.149060 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad655428-de9e-4492-b4f9-1ed53b09b4dd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:23.150486 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad655428-de9e-4492-b4f9-1ed53b09b4dd None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:23.161734 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:23.172502 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:23.201076 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004d2489-ba06-40d3-8df7-0c3efef730c3 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:23.235474 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5708bea6-05cc-4bb3-92a1-624e7906b4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:23.236091 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5708bea6-05cc-4bb3-92a1-624e7906b4d2 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:23.236375 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5708bea6-05cc-4bb3-92a1-624e7906b4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:23.236682 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5708bea6-05cc-4bb3-92a1-624e7906b4d2 None admin] PATH_INFO: `/v3/services` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:23.237319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5708bea6-05cc-4bb3-92a1-624e7906b4d2 None admin] RBAC: Adding query filter params (name=image) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:23.237628 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5708bea6-05cc-4bb3-92a1-624e7906b4d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:23.239037 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5708bea6-05cc-4bb3-92a1-624e7906b4d2 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:23.243927 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 523/1027] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:25:23 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:25:23.250570 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5708bea6-05cc-4bb3-92a1-624e7906b4d2 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:23.265813 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 505/1028] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Aug 24 05:25:22 2022] POST /identity/v3/users => generated 285 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:25:23.378690 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5708bea6-05cc-4bb3-92a1-624e7906b4d2 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:23.417035 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ea253-7c2a-421c-9f0b-dd0aa8b831ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:23.417707 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ea253-7c2a-421c-9f0b-dd0aa8b831ad None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:23.417995 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ea253-7c2a-421c-9f0b-dd0aa8b831ad None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:23.418296 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ea253-7c2a-421c-9f0b-dd0aa8b831ad None admin] PATH_INFO: `/v3/services` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:23.419737 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34ea253-7c2a-421c-9f0b-dd0aa8b831ad None admin] RBAC: Adding query filter params (type=image) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:23.419737 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34ea253-7c2a-421c-9f0b-dd0aa8b831ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:23.420816 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34ea253-7c2a-421c-9f0b-dd0aa8b831ad None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:23.426319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 524/1029] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:25:23 2022] GET /identity/v3/services?type=image => generated 350 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:23.432684 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad655428-de9e-4492-b4f9-1ed53b09b4dd None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:23.539804 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad655428-de9e-4492-b4f9-1ed53b09b4dd None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:23.574904 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f460b0-b168-4df3-aa64-1ac74c13dbd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:23.575740 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f460b0-b168-4df3-aa64-1ac74c13dbd5 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:23.575969 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f460b0-b168-4df3-aa64-1ac74c13dbd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:23.576241 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f460b0-b168-4df3-aa64-1ac74c13dbd5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:23.576963 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f460b0-b168-4df3-aa64-1ac74c13dbd5 None admin] RBAC: Adding query filter params (interface=public, service_id=81a6461bf6a94608be55e9f04ebd7b7c, region_id=RegionOne) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:23.577272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f460b0-b168-4df3-aa64-1ac74c13dbd5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:23.578638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f460b0-b168-4df3-aa64-1ac74c13dbd5 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:23.588384 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 506/1030] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Aug 24 05:25:23 2022] GET /identity/v3/endpoints?service_id=81a6461bf6a94608be55e9f04ebd7b7c&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:23.589716 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ea253-7c2a-421c-9f0b-dd0aa8b831ad None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:23.593808 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:23.699854 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ea253-7c2a-421c-9f0b-dd0aa8b831ad None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:23.713474 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70814158-5023-4a5f-b4a2-9c0470f05215 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:23.713771 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70814158-5023-4a5f-b4a2-9c0470f05215 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:23.714058 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70814158-5023-4a5f-b4a2-9c0470f05215 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:23.715392 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 507/1031] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:25:23.720667 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193edebe-0016-4e23-96cb-7ff182be5415 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:23.720990 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193edebe-0016-4e23-96cb-7ff182be5415 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:23.721226 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193edebe-0016-4e23-96cb-7ff182be5415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:23.739762 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98166a94-aea3-4ebc-904e-561ba57fe90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:23.740195 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-193edebe-0016-4e23-96cb-7ff182be5415 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:25:23.740447 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98166a94-aea3-4ebc-904e-561ba57fe90d None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:23.740738 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98166a94-aea3-4ebc-904e-561ba57fe90d None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:23.741017 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98166a94-aea3-4ebc-904e-561ba57fe90d None admin] PATH_INFO: `/v3/services` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:23.741478 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-193edebe-0016-4e23-96cb-7ff182be5415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:25:23.741657 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98166a94-aea3-4ebc-904e-561ba57fe90d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:23.742990 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98166a94-aea3-4ebc-904e-561ba57fe90d None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:23.758583 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 525/1032] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Aug 24 05:25:23 2022] GET /identity/v3/services => generated 1957 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:25:23.814925 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:23.834410 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 508/1033] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:25:23 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:25:24.107594 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98166a94-aea3-4ebc-904e-561ba57fe90d None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:24.174984 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:24.219683 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98166a94-aea3-4ebc-904e-561ba57fe90d None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:24.255208 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c35b4f7-7421-45ba-aea3-ed9c3ecf9498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:24.255846 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35b4f7-7421-45ba-aea3-ed9c3ecf9498 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:24.256130 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35b4f7-7421-45ba-aea3-ed9c3ecf9498 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:24.256427 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35b4f7-7421-45ba-aea3-ed9c3ecf9498 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:24.267309 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c35b4f7-7421-45ba-aea3-ed9c3ecf9498 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:25:24.272494 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-2c35b4f7-7421-45ba-aea3-ed9c3ecf9498 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 05:25:24.272494 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:24.272494 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:24.272494 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:24.272494 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:24.272494 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:24.272494 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:24.272494 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:24.272494 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:24.272494 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:24.272494 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:24.272494 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:24.272494 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:25:24.272494 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:25:24.272494 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:25:24.272494 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:25:24.273428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:25:24.273428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:25:24.273428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:25:24.273428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:25:24.273428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:24.273428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:24.273428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:24.273428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:24.273428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:24.273428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:24.273428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:24.273428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:24.273428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:24.273428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:24.273428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:24.273428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:24.274334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:24.274334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:24.274334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:24.274334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:24.274334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:25:24.274334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:25:24.274334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:25:24.274334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:25:24.274334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:25:24.274334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:25:24.274334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 05:25:24.274334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:24.276776 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 526/1034] 10.222.0.15 () {56 vars in 1120 bytes} [Wed Aug 24 05:25:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:25:24.279642 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193edebe-0016-4e23-96cb-7ff182be5415 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:24.385428 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193edebe-0016-4e23-96cb-7ff182be5415 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:24.421697 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f63335-6978-4958-924b-778910c0d95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:24.422330 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f63335-6978-4958-924b-778910c0d95a None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:24.422613 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f63335-6978-4958-924b-778910c0d95a None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:24.422886 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f63335-6978-4958-924b-778910c0d95a None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:24.423575 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f63335-6978-4958-924b-778910c0d95a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:24.423883 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f63335-6978-4958-924b-778910c0d95a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:24.425330 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f63335-6978-4958-924b-778910c0d95a None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:24.440437 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 509/1035] 10.222.0.15 () {56 vars in 1115 bytes} [Wed Aug 24 05:25:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:24.441488 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c35b4f7-7421-45ba-aea3-ed9c3ecf9498 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:24.545858 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c35b4f7-7421-45ba-aea3-ed9c3ecf9498 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:24.571167 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:24.580199 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92eda5-9ad7-4a72-a669-ea7174d6ddee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:24.580892 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92eda5-9ad7-4a72-a669-ea7174d6ddee None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:24.581174 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92eda5-9ad7-4a72-a669-ea7174d6ddee None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:24.581448 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92eda5-9ad7-4a72-a669-ea7174d6ddee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:24.591921 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed92eda5-9ad7-4a72-a669-ea7174d6ddee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:24.593385 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed92eda5-9ad7-4a72-a669-ea7174d6ddee None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:24.608822 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-ed92eda5-9ad7-4a72-a669-ea7174d6ddee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:25:24.608822 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:24.608822 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:24.608822 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:24.608822 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:24.608822 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:24.608822 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:24.608822 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:24.608822 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:24.608822 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:24.608822 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:24.608822 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:24.608822 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:25:24.608822 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:25:24.608822 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:25:24.608822 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:25:24.609890 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:24.609890 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:24.609890 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:24.609890 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:24.609890 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:24.609890 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:24.609890 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:24.609890 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:24.609890 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:24.609890 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:24.609890 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:24.609890 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:24.609890 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:24.609890 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:24.609890 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:24.609890 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:24.609890 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:25:24.615896 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:25:24.615896 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:25:24.615896 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:24.615896 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 527/1036] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:25:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:25:24.617793 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f63335-6978-4958-924b-778910c0d95a None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:24.673200 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:24.730538 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f63335-6978-4958-924b-778910c0d95a None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:24.779998 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795023d1-56a3-44b9-afb3-611ff9601bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:24.780856 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795023d1-56a3-44b9-afb3-611ff9601bc0 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:24.781636 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795023d1-56a3-44b9-afb3-611ff9601bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:24.782202 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795023d1-56a3-44b9-afb3-611ff9601bc0 None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:24.783004 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795023d1-56a3-44b9-afb3-611ff9601bc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:24.783480 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795023d1-56a3-44b9-afb3-611ff9601bc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:24.785672 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795023d1-56a3-44b9-afb3-611ff9601bc0 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:24.804795 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 510/1037] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:25:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:24.811013 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92eda5-9ad7-4a72-a669-ea7174d6ddee None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:24.920025 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92eda5-9ad7-4a72-a669-ea7174d6ddee None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:24.954318 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1229e31f-a0ef-4527-bb33-4b76052e8793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:24.954872 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229e31f-a0ef-4527-bb33-4b76052e8793 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:24.955148 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229e31f-a0ef-4527-bb33-4b76052e8793 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:24.955419 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229e31f-a0ef-4527-bb33-4b76052e8793 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:24.963078 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1229e31f-a0ef-4527-bb33-4b76052e8793 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:24.964547 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1229e31f-a0ef-4527-bb33-4b76052e8793 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:24.976347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-1229e31f-a0ef-4527-bb33-4b76052e8793 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:25:24.976347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:24.976347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:24.976347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:24.976347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:24.976347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:24.976347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:24.976347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:24.976347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:24.976347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:24.976347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:24.976347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:24.976347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:25:24.976347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:25:24.976347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:25:24.976347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:25:24.977384 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:24.977384 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:24.977384 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:25:24.977384 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:25:24.977384 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:25:24.977384 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:25:24.977384 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:24.977384 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:24.977384 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:24.977384 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:24.977384 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:24.977384 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:24.977384 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:24.977384 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:24.977384 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:24.977384 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:24.977384 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:24.977384 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:24.984072 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:24.984072 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:24.984072 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:25:24.984072 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:25:24.984072 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:25:24.984072 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:25:24.984072 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:25:24.984072 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:25:24.984072 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:25:24.984072 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:24.984072 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 528/1038] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Aug 24 05:25:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:25:24.984713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795023d1-56a3-44b9-afb3-611ff9601bc0 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:25.089880 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795023d1-56a3-44b9-afb3-611ff9601bc0 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:25.099978 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:25.134622 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4803d4dd-8ca8-4927-ad3f-85cc8edcbfcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:25.135404 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803d4dd-8ca8-4927-ad3f-85cc8edcbfcb None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:25.135640 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803d4dd-8ca8-4927-ad3f-85cc8edcbfcb None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:25.135915 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803d4dd-8ca8-4927-ad3f-85cc8edcbfcb None admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:25.136763 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4803d4dd-8ca8-4927-ad3f-85cc8edcbfcb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:25.137112 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4803d4dd-8ca8-4927-ad3f-85cc8edcbfcb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:25.139143 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4803d4dd-8ca8-4927-ad3f-85cc8edcbfcb None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:25.167929 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 511/1039] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Aug 24 05:25:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:25.170125 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1229e31f-a0ef-4527-bb33-4b76052e8793 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:25.215172 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:25.277598 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1229e31f-a0ef-4527-bb33-4b76052e8793 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:25.279869 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccacd2a6-f02f-4d1e-ba72-f8f6450921d2 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:25.280160 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccacd2a6-f02f-4d1e-ba72-f8f6450921d2 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:25.280461 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccacd2a6-f02f-4d1e-ba72-f8f6450921d2 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:25.282211 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 512/1040] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:25:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:25:25.287391 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3adf0-a2f1-4e01-8c8e-496246838f84 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:25.287677 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3adf0-a2f1-4e01-8c8e-496246838f84 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:25.287952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3adf0-a2f1-4e01-8c8e-496246838f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:25.306512 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-cdf3adf0-a2f1-4e01-8c8e-496246838f84 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:25:25.307705 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-cdf3adf0-a2f1-4e01-8c8e-496246838f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:25:25.316413 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2d8ca-ced6-4a84-958e-1a6de3d902ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:25.317034 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2d8ca-ced6-4a84-958e-1a6de3d902ad None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:25.317317 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2d8ca-ced6-4a84-958e-1a6de3d902ad None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:25.317592 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2d8ca-ced6-4a84-958e-1a6de3d902ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:25.381023 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 513/1041] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:25:25 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:25:25.407201 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d2d8ca-ced6-4a84-958e-1a6de3d902ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:25.408690 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d2d8ca-ced6-4a84-958e-1a6de3d902ad None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:25.451290 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:25.513635 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 529/1042] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:25:25 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:25:25.519234 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3adf0-a2f1-4e01-8c8e-496246838f84 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:25.617710 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:25.624307 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3adf0-a2f1-4e01-8c8e-496246838f84 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:25.652804 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2d8ca-ced6-4a84-958e-1a6de3d902ad None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:25.661425 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09946e6b-7604-407e-bd3d-7adab926131d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:25.662064 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09946e6b-7604-407e-bd3d-7adab926131d None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:25.662372 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09946e6b-7604-407e-bd3d-7adab926131d None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:25.662653 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09946e6b-7604-407e-bd3d-7adab926131d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:25.674099 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09946e6b-7604-407e-bd3d-7adab926131d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:25.675551 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09946e6b-7604-407e-bd3d-7adab926131d None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:25.692127 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-09946e6b-7604-407e-bd3d-7adab926131d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:25:25.692127 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:25.692127 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:25.692127 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:25.692127 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:25.692127 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:25.692127 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:25.692127 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:25.692127 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:25.692127 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:25.692127 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:25.692127 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:25.692127 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:25:25.692127 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:25:25.692127 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:25:25.692127 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:25:25.693407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:25.693407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:25.693407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:25.693407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:25.693407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:25.693407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:25.693407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:25.693407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:25.693407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:25.693407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:25.693407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:25.693407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:25.693407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:25.693407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:25.693407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:25.693407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:25.693407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:25:25.694562 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:25:25.694562 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:25:25.694562 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:25:25.694562 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 514/1043] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:25:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:25:25.698921 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09946e6b-7604-407e-bd3d-7adab926131d None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:25.777787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2d8ca-ced6-4a84-958e-1a6de3d902ad None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:25.813620 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09946e6b-7604-407e-bd3d-7adab926131d None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:25.814574 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef7e23-3caa-4b34-b811-26eba440e84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:25.815215 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef7e23-3caa-4b34-b811-26eba440e84f None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:25.815497 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef7e23-3caa-4b34-b811-26eba440e84f None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:25.816508 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef7e23-3caa-4b34-b811-26eba440e84f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:25.823390 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:25.828876 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ef7e23-3caa-4b34-b811-26eba440e84f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:25:25.833712 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-62ef7e23-3caa-4b34-b811-26eba440e84f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:25:25.833712 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:25.833712 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:25.833712 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:25.833712 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:25.833712 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:25.833712 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:25.833712 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:25.833712 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:25.833712 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:25.833712 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:25.833712 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:25.833712 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:25:25.833712 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:25:25.833712 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:25:25.833712 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:25:25.834684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:25:25.834684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:25:25.834684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:25:25.834684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:25:25.834684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:25.834684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:25.834684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:25.834684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:25.834684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:25.834684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:25.834684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:25.834684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:25.834684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:25.834684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:25.834684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:25.834684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:25.835579 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:25.835579 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:25.835579 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:25.835579 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:25.835579 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:25:25.835579 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:25:25.835579 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:25:25.835579 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:25:25.835579 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:25:25.835579 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:25:25.835579 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:25:25.835579 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:25.835579 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 530/1044] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Aug 24 05:25:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:25:25.836560 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7d173-69ea-4626-b155-76b876dbedff None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:25.836819 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7d173-69ea-4626-b155-76b876dbedff None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:25.837250 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7d173-69ea-4626-b155-76b876dbedff None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:25.838528 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 531/1045] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:25:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:25:25.842020 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def7d173-69ea-4626-b155-76b876dbedff None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:25.852329 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8cab7b-2b34-44b0-b444-5dfbeeffa592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:25.853136 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8cab7b-2b34-44b0-b444-5dfbeeffa592 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:25.853424 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8cab7b-2b34-44b0-b444-5dfbeeffa592 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:25.853699 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8cab7b-2b34-44b0-b444-5dfbeeffa592 None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:25.854341 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8cab7b-2b34-44b0-b444-5dfbeeffa592 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:25.854650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8cab7b-2b34-44b0-b444-5dfbeeffa592 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:25.856053 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8cab7b-2b34-44b0-b444-5dfbeeffa592 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:25.868488 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 515/1046] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:25:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:25:25.869697 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b7c40-a7b6-42d8-afe5-614289ca180c None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:25.870043 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b7c40-a7b6-42d8-afe5-614289ca180c None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:25.870335 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b7c40-a7b6-42d8-afe5-614289ca180c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:25.890749 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-ba9b7c40-a7b6-42d8-afe5-614289ca180c None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:25:25.891964 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-ba9b7c40-a7b6-42d8-afe5-614289ca180c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:25:25.894042 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:25.947147 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def7d173-69ea-4626-b155-76b876dbedff None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:25.960998 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 516/1047] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:25:25 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:25:25.961842 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9b7c40-a7b6-42d8-afe5-614289ca180c None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:25.982314 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be1755f-123f-464c-b0d6-83b5780fd0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:25.982920 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be1755f-123f-464c-b0d6-83b5780fd0bc None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:25.983204 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be1755f-123f-464c-b0d6-83b5780fd0bc None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:25.983482 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be1755f-123f-464c-b0d6-83b5780fd0bc None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:25.984144 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be1755f-123f-464c-b0d6-83b5780fd0bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:25.984479 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be1755f-123f-464c-b0d6-83b5780fd0bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:25.985960 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be1755f-123f-464c-b0d6-83b5780fd0bc None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:25.999787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 532/1048] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:25:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:25:26.000955 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be1755f-123f-464c-b0d6-83b5780fd0bc None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:26.066365 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:26.071916 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9b7c40-a7b6-42d8-afe5-614289ca180c None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:26.109556 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15814fa3-3cc8-4c27-895f-c2838cf1cf3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:26.110188 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15814fa3-3cc8-4c27-895f-c2838cf1cf3b None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:26.110314 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be1755f-123f-464c-b0d6-83b5780fd0bc None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:26.110507 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15814fa3-3cc8-4c27-895f-c2838cf1cf3b None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:26.110741 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15814fa3-3cc8-4c27-895f-c2838cf1cf3b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:26.117655 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15814fa3-3cc8-4c27-895f-c2838cf1cf3b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:26.119100 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15814fa3-3cc8-4c27-895f-c2838cf1cf3b None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:26.120978 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:26.131963 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-15814fa3-3cc8-4c27-895f-c2838cf1cf3b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:25:26.131963 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:26.131963 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:26.131963 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:26.131963 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:26.131963 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:26.131963 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:26.131963 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:26.131963 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:26.131963 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:26.131963 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:26.131963 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:26.131963 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:25:26.131963 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:25:26.131963 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:25:26.138299 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:25:26.138299 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:26.138299 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:26.138299 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:26.138299 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:26.138299 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:26.138299 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:26.138299 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:26.138299 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:26.138299 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:26.138299 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:26.138299 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:26.138299 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:26.138299 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:26.138299 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:26.138299 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:26.138299 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:26.139203 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:25:26.139203 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:25:26.139203 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:25:26.139203 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:25:26.139203 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:25:26.139203 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:25:26.139203 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:25:26.139203 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:25:26.139203 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 517/1049] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:25:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:25:26.139203 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15814fa3-3cc8-4c27-895f-c2838cf1cf3b None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:26.151123 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c611e358-dd64-4f41-b859-7f5ef8510d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:26.151795 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c611e358-dd64-4f41-b859-7f5ef8510d45 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:26.152041 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c611e358-dd64-4f41-b859-7f5ef8510d45 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:26.152316 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c611e358-dd64-4f41-b859-7f5ef8510d45 None admin] PATH_INFO: `/v3/users/2ff9edade73940bf8b6ee80d26abe8c6` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:26.168176 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c611e358-dd64-4f41-b859-7f5ef8510d45 None admin] RBAC: Authorizing `identity:get_user(user_id=2ff9edade73940bf8b6ee80d26abe8c6)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:26.170120 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c611e358-dd64-4f41-b859-7f5ef8510d45 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:26.176650 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 533/1050] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:25:25 2022] GET /identity/v3/users/2ff9edade73940bf8b6ee80d26abe8c6 => generated 285 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:26.180065 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c611e358-dd64-4f41-b859-7f5ef8510d45 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:26.254748 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15814fa3-3cc8-4c27-895f-c2838cf1cf3b None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:26.288578 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c611e358-dd64-4f41-b859-7f5ef8510d45 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:26.289492 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ae6a7-8f43-41c9-9a20-421378517977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:26.290105 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ae6a7-8f43-41c9-9a20-421378517977 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:26.290381 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ae6a7-8f43-41c9-9a20-421378517977 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:26.290651 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ae6a7-8f43-41c9-9a20-421378517977 None admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:26.291338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08ae6a7-8f43-41c9-9a20-421378517977 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:26.291640 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08ae6a7-8f43-41c9-9a20-421378517977 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:26.293096 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08ae6a7-8f43-41c9-9a20-421378517977 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:26.304963 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 518/1051] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Aug 24 05:25:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:26.305768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ae6a7-8f43-41c9-9a20-421378517977 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:26.310426 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:26.327348 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c8de26-83c3-4481-b5da-9881437be5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:26.327878 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8de26-83c3-4481-b5da-9881437be5a6 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:26.328158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8de26-83c3-4481-b5da-9881437be5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:26.328489 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8de26-83c3-4481-b5da-9881437be5a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:26.367476 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:26.396379 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c8de26-83c3-4481-b5da-9881437be5a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:26.398000 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c8de26-83c3-4481-b5da-9881437be5a6 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:26.410792 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ae6a7-8f43-41c9-9a20-421378517977 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:26.448232 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d778d84d-c427-432c-ad88-bee9d2777797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:26.448961 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778d84d-c427-432c-ad88-bee9d2777797 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:26.449230 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778d84d-c427-432c-ad88-bee9d2777797 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:26.449499 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778d84d-c427-432c-ad88-bee9d2777797 None admin] PATH_INFO: `/v3/services/image` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:26.450782 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d778d84d-c427-432c-ad88-bee9d2777797 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:26.452847 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d778d84d-c427-432c-ad88-bee9d2777797 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:26.461935 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-d778d84d-c427-432c-ad88-bee9d2777797 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 05:25:26.461935 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:26.461935 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:25:26.461935 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:25:26.461935 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:25:26.461935 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:25:26.461935 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:25:26.461935 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:25:26.461935 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 05:25:26.461935 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:25:26.461935 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:25:26.461935 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:25:26.461935 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:26.461935 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:26.461935 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:26.461935 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:26.461935 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:26.463179 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:26.463179 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:26.463179 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:26.463179 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:26.463179 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:26.463179 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:26.463179 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:25:26.463179 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:25:26.463179 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:25:26.463179 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:25:26.463179 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:26.463179 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:26.463179 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:26.463179 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:26.463179 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:26.463179 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:26.464129 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:26.464129 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:26.464129 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:26.464129 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:26.464129 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:26.464129 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:26.464129 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:26.464129 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:26.464129 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:26.464129 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:26.464129 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:25:26.464129 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:25:26.464129 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 05:25:26.464129 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:25:26.464129 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 519/1052] 10.222.0.15 () {56 vars in 1105 bytes} [Wed Aug 24 05:25:26 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:25:26.464985 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d778d84d-c427-432c-ad88-bee9d2777797 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:26.504476 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 534/1053] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:25:26 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 05:25:26.505628 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c8de26-83c3-4481-b5da-9881437be5a6 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:26.526825 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:26.570841 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d778d84d-c427-432c-ad88-bee9d2777797 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:26.593848 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:26.605571 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a4157-5f1f-436c-ba55-5c474321456f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:26.606880 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a4157-5f1f-436c-ba55-5c474321456f None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:26.607438 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a4157-5f1f-436c-ba55-5c474321456f None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:26.607438 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a4157-5f1f-436c-ba55-5c474321456f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:26.608167 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783a4157-5f1f-436c-ba55-5c474321456f None admin] RBAC: Adding query filter params (role.id=719a48d7876b4ffa9489dcc5d87cfbad, scope.project.id=59592706ae9442e08353158c625e9581, user.id=fbecffb800584c5786813accd061421b) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:26.608508 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783a4157-5f1f-436c-ba55-5c474321456f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:26.608644 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c8de26-83c3-4481-b5da-9881437be5a6 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:26.609947 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783a4157-5f1f-436c-ba55-5c474321456f None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:26.615450 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 520/1054] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:25:26 2022] GET /identity/v3/role_assignments?user.id=fbecffb800584c5786813accd061421b&scope.project.id=59592706ae9442e08353158c625e9581&role.id=719a48d7876b4ffa9489dcc5d87cfbad => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:26.617669 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a4157-5f1f-436c-ba55-5c474321456f None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:26.649544 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c066b2-6de0-4d6f-a18a-ddf0245f0683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:26.650167 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c066b2-6de0-4d6f-a18a-ddf0245f0683 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:26.650446 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c066b2-6de0-4d6f-a18a-ddf0245f0683 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:26.650744 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c066b2-6de0-4d6f-a18a-ddf0245f0683 None admin] PATH_INFO: `/v3/services` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:26.651366 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c066b2-6de0-4d6f-a18a-ddf0245f0683 None admin] RBAC: Adding query filter params (name=image) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:26.651672 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c066b2-6de0-4d6f-a18a-ddf0245f0683 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:26.653142 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c066b2-6de0-4d6f-a18a-ddf0245f0683 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:26.663990 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 535/1055] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:25:26 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:26.665549 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c066b2-6de0-4d6f-a18a-ddf0245f0683 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:26.751523 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a4157-5f1f-436c-ba55-5c474321456f None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:26.765391 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:26.786389 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c066b2-6de0-4d6f-a18a-ddf0245f0683 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:26.802842 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9603d199-1e69-4bc8-965e-4b422c3c4d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:26.803493 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9603d199-1e69-4bc8-965e-4b422c3c4d5d None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:26.803767 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9603d199-1e69-4bc8-965e-4b422c3c4d5d None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:26.804041 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9603d199-1e69-4bc8-965e-4b422c3c4d5d None admin] PATH_INFO: `/v3/projects/42f99470856a442cad8caccfa62dff6a` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:26.805261 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9603d199-1e69-4bc8-965e-4b422c3c4d5d None admin] RBAC: Authorizing `identity:get_project(project_id=42f99470856a442cad8caccfa62dff6a)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:26.806791 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9603d199-1e69-4bc8-965e-4b422c3c4d5d None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:26.808909 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 521/1056] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:25:26 2022] GET /identity/v3/projects/42f99470856a442cad8caccfa62dff6a => generated 311 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:26.816040 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9603d199-1e69-4bc8-965e-4b422c3c4d5d None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:26.836496 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9680d-6401-4168-a383-73c3cf344902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:26.837129 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9680d-6401-4168-a383-73c3cf344902 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:26.837414 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9680d-6401-4168-a383-73c3cf344902 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:26.837692 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9680d-6401-4168-a383-73c3cf344902 None admin] PATH_INFO: `/v3/services` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:26.838334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b9680d-6401-4168-a383-73c3cf344902 None admin] RBAC: Adding query filter params (type=image) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:26.838642 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b9680d-6401-4168-a383-73c3cf344902 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:26.840021 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b9680d-6401-4168-a383-73c3cf344902 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:26.846037 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 536/1057] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:25:26 2022] GET /identity/v3/services?type=image => generated 350 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:25:26.851765 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9680d-6401-4168-a383-73c3cf344902 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:26.880717 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:26.931272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9603d199-1e69-4bc8-965e-4b422c3c4d5d None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:26.960331 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9680d-6401-4168-a383-73c3cf344902 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:26.972560 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671fd232-7b87-465d-b406-7b6efff617f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:26.973144 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671fd232-7b87-465d-b406-7b6efff617f7 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:26.973445 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671fd232-7b87-465d-b406-7b6efff617f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:26.973703 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671fd232-7b87-465d-b406-7b6efff617f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:26.974436 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671fd232-7b87-465d-b406-7b6efff617f7 None admin] RBAC: Adding query filter params (role.id=e013ad4d436f4cb0a9bf7debb7d13dc7, scope.project.id=42f99470856a442cad8caccfa62dff6a, user.id=2ff9edade73940bf8b6ee80d26abe8c6) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:26.974754 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671fd232-7b87-465d-b406-7b6efff617f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:26.976181 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671fd232-7b87-465d-b406-7b6efff617f7 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:26.982291 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 522/1058] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:25:26 2022] GET /identity/v3/role_assignments?user.id=2ff9edade73940bf8b6ee80d26abe8c6&scope.project.id=42f99470856a442cad8caccfa62dff6a&role.id=e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:27.011688 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83a5849-15cc-436d-8c98-945adbb871be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:27.013187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a5849-15cc-436d-8c98-945adbb871be None admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:27.013324 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a5849-15cc-436d-8c98-945adbb871be None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:27.013563 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a5849-15cc-436d-8c98-945adbb871be None admin] PATH_INFO: `/v3/endpoints` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:27.014247 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83a5849-15cc-436d-8c98-945adbb871be None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:27.015707 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83a5849-15cc-436d-8c98-945adbb871be None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:27.132608 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 537/1059] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:25:26 2022] POST /identity/v3/endpoints => generated 327 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:25:28.540449 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e6363-130c-4a40-b3ff-e321127ae5ef None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:28.540957 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e6363-130c-4a40-b3ff-e321127ae5ef None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:28.540957 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e6363-130c-4a40-b3ff-e321127ae5ef None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:28.542392 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 523/1060] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:25:28.547592 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea07fbd-559b-47aa-bac4-7eb257b1c54c None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:28.548068 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea07fbd-559b-47aa-bac4-7eb257b1c54c None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:28.548164 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea07fbd-559b-47aa-bac4-7eb257b1c54c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:28.554508 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:28.567328 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-dea07fbd-559b-47aa-bac4-7eb257b1c54c None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:25:28.568198 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-dea07fbd-559b-47aa-bac4-7eb257b1c54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:25:28.647899 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 538/1061] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:25:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:25:28.917608 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e6363-130c-4a40-b3ff-e321127ae5ef None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:28.923413 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5748ab8-9cbf-4aac-996e-fdfcb0421a86 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:28.923702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5748ab8-9cbf-4aac-996e-fdfcb0421a86 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:28.923978 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5748ab8-9cbf-4aac-996e-fdfcb0421a86 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:28.925396 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 539/1062] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:25:28.930711 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc3e40-9f8a-48c3-8e3e-d465525513c7 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:28.931017 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc3e40-9f8a-48c3-8e3e-d465525513c7 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:28.931274 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc3e40-9f8a-48c3-8e3e-d465525513c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:28.951467 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-0fcc3e40-9f8a-48c3-8e3e-d465525513c7 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:25:28.952976 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-0fcc3e40-9f8a-48c3-8e3e-d465525513c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:25:29.039593 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 540/1063] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:25:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:25:29.044389 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e6363-130c-4a40-b3ff-e321127ae5ef None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:29.064134 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:29.081317 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b48de-7d03-4c54-98ba-8dc38f5d674d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:29.081957 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b48de-7d03-4c54-98ba-8dc38f5d674d None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:29.082236 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b48de-7d03-4c54-98ba-8dc38f5d674d None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:29.082511 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b48de-7d03-4c54-98ba-8dc38f5d674d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:29.082731 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ed94e-1efa-49c6-92d8-04262ba0cfbf None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:29.085412 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ed94e-1efa-49c6-92d8-04262ba0cfbf None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:29.085412 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ed94e-1efa-49c6-92d8-04262ba0cfbf None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:29.085412 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 541/1064] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:25:29.089855 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f8f71-9f26-47d7-b332-6319d83e3161 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:29.090147 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f8f71-9f26-47d7-b332-6319d83e3161 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:29.090762 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f8f71-9f26-47d7-b332-6319d83e3161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:29.093997 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2b48de-7d03-4c54-98ba-8dc38f5d674d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:25:29.099759 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-ca2b48de-7d03-4c54-98ba-8dc38f5d674d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 05:25:29.099759 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:29.099759 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:29.099759 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:29.099759 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:29.099759 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:29.099759 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:29.099759 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:29.099759 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:29.099759 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:29.099759 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:29.099759 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:29.099759 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:25:29.099759 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:25:29.099759 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:25:29.099759 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:25:29.100938 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:25:29.100938 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:25:29.100938 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:25:29.100938 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:25:29.100938 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:29.100938 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:29.100938 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:29.100938 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:29.100938 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:29.100938 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:29.100938 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:29.100938 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:29.100938 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:29.100938 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:29.100938 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:29.100938 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:29.102284 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:29.102284 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:29.102284 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:29.102284 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:29.102284 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:25:29.102284 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:25:29.102284 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:25:29.102284 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:25:29.102284 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:25:29.102284 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:25:29.102284 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 05:25:29.102284 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:25:29.102284 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 524/1065] 10.222.0.15 () {56 vars in 1120 bytes} [Wed Aug 24 05:25:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:25:29.109530 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b48de-7d03-4c54-98ba-8dc38f5d674d None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:29.113438 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-ca3f8f71-9f26-47d7-b332-6319d83e3161 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:25:29.114730 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-ca3f8f71-9f26-47d7-b332-6319d83e3161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:25:29.179203 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:29.198255 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 542/1066] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:25:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:25:29.233749 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b48de-7d03-4c54-98ba-8dc38f5d674d None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:29.269426 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69e440-fee1-451f-9ffb-4b3353296d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:29.270040 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69e440-fee1-451f-9ffb-4b3353296d93 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:29.270323 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69e440-fee1-451f-9ffb-4b3353296d93 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:29.270596 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69e440-fee1-451f-9ffb-4b3353296d93 None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:29.271254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d69e440-fee1-451f-9ffb-4b3353296d93 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:29.271556 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d69e440-fee1-451f-9ffb-4b3353296d93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:29.272951 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d69e440-fee1-451f-9ffb-4b3353296d93 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:29.282193 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 525/1067] 10.222.0.15 () {56 vars in 1115 bytes} [Wed Aug 24 05:25:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:25:29.288149 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f8f71-9f26-47d7-b332-6319d83e3161 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:29.314381 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69e440-fee1-451f-9ffb-4b3353296d93 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:29.357948 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:29.406868 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f8f71-9f26-47d7-b332-6319d83e3161 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:29.418285 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69e440-fee1-451f-9ffb-4b3353296d93 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:29.456762 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45b3478-63dc-4349-9b1d-587f544f61c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:29.457504 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b3478-63dc-4349-9b1d-587f544f61c4 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:29.457794 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b3478-63dc-4349-9b1d-587f544f61c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:29.458070 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b3478-63dc-4349-9b1d-587f544f61c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:29.460288 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc128f-9569-45da-927c-3eb0ba88c973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:29.460966 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc128f-9569-45da-927c-3eb0ba88c973 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:29.461250 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc128f-9569-45da-927c-3eb0ba88c973 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:29.461532 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc128f-9569-45da-927c-3eb0ba88c973 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:29.469639 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45b3478-63dc-4349-9b1d-587f544f61c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:29.471364 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45b3478-63dc-4349-9b1d-587f544f61c4 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:29.473924 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bc128f-9569-45da-927c-3eb0ba88c973 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:25:29.478654 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-d3bc128f-9569-45da-927c-3eb0ba88c973 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:25:29.478654 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:29.478654 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:29.478654 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:29.478654 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:29.478654 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:29.478654 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:29.478654 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:29.478654 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:29.478654 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:29.478654 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:29.478654 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:29.478654 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:25:29.478654 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:25:29.478654 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:25:29.478654 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:25:29.479653 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:25:29.479653 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:25:29.479653 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:25:29.479653 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:25:29.479653 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:29.479653 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:29.479653 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:29.479653 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:29.479653 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:29.479653 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:29.479653 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:29.479653 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:29.479653 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:29.479653 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:29.479653 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:29.479653 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:29.480527 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:29.480527 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:29.480527 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:29.480527 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:29.480527 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:25:29.480527 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:25:29.480527 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:25:29.480527 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:25:29.480527 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:25:29.480527 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:25:29.480527 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:25:29.480527 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:25:29.480527 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 526/1068] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Aug 24 05:25:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:25:29.481264 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc128f-9569-45da-927c-3eb0ba88c973 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:29.491579 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-c45b3478-63dc-4349-9b1d-587f544f61c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:25:29.491579 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:29.491579 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:29.491579 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:29.491579 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:29.491579 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:29.491579 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:29.491579 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:29.491579 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:29.491579 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:29.491579 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:29.491579 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:29.491579 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:25:29.491579 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:25:29.491579 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:25:29.491579 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:25:29.499403 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:29.499403 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:29.499403 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:29.499403 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:29.499403 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:29.499403 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:29.499403 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:29.499403 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:29.499403 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:29.499403 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:29.499403 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:29.499403 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:29.499403 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:29.499403 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:29.499403 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:29.499403 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:29.499403 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:25:29.500390 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:25:29.500390 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:25:29.500390 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:29.500390 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 543/1069] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:25:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:25:29.500390 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45b3478-63dc-4349-9b1d-587f544f61c4 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:29.536324 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:29.589049 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:29.592563 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc128f-9569-45da-927c-3eb0ba88c973 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:29.595618 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45b3478-63dc-4349-9b1d-587f544f61c4 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:29.628650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a48788-5d29-426e-9952-30ed7a6377d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:29.629263 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a48788-5d29-426e-9952-30ed7a6377d3 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:29.629548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a48788-5d29-426e-9952-30ed7a6377d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:29.629825 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a48788-5d29-426e-9952-30ed7a6377d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:29.631114 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf732800-f839-4a82-968a-b9feb5a192ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:29.631728 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf732800-f839-4a82-968a-b9feb5a192ee None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:29.632011 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf732800-f839-4a82-968a-b9feb5a192ee None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:29.632286 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf732800-f839-4a82-968a-b9feb5a192ee None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:29.633013 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf732800-f839-4a82-968a-b9feb5a192ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:29.633321 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf732800-f839-4a82-968a-b9feb5a192ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:29.634821 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf732800-f839-4a82-968a-b9feb5a192ee None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:29.644463 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 544/1070] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:25:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:29.645321 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf732800-f839-4a82-968a-b9feb5a192ee None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:29.702249 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a48788-5d29-426e-9952-30ed7a6377d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:29.703689 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a48788-5d29-426e-9952-30ed7a6377d3 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:29.745927 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf732800-f839-4a82-968a-b9feb5a192ee None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:29.747436 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:29.791329 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8850fc12-6456-4159-b975-f4f2c5904b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:29.791945 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8850fc12-6456-4159-b975-f4f2c5904b69 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:29.792223 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8850fc12-6456-4159-b975-f4f2c5904b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:29.792533 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8850fc12-6456-4159-b975-f4f2c5904b69 None admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:29.793179 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8850fc12-6456-4159-b975-f4f2c5904b69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:29.793567 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8850fc12-6456-4159-b975-f4f2c5904b69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:29.794938 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8850fc12-6456-4159-b975-f4f2c5904b69 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:29.806335 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 545/1071] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:25:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:29.807231 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8850fc12-6456-4159-b975-f4f2c5904b69 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:29.809351 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:29.816453 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 527/1072] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:25:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:25:29.816453 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a48788-5d29-426e-9952-30ed7a6377d3 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:29.913091 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8850fc12-6456-4159-b975-f4f2c5904b69 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:29.918739 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a48788-5d29-426e-9952-30ed7a6377d3 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:29.948854 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040e508-b05d-4a99-8790-7fff9b5bd630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:29.949484 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040e508-b05d-4a99-8790-7fff9b5bd630 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:29.950487 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040e508-b05d-4a99-8790-7fff9b5bd630 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:29.950745 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040e508-b05d-4a99-8790-7fff9b5bd630 None admin] PATH_INFO: `/v3/users/2ff9edade73940bf8b6ee80d26abe8c6` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:29.951995 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f040e508-b05d-4a99-8790-7fff9b5bd630 None admin] RBAC: Authorizing `identity:get_user(user_id=2ff9edade73940bf8b6ee80d26abe8c6)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:29.953454 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f040e508-b05d-4a99-8790-7fff9b5bd630 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:29.955201 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 546/1073] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:25:29 2022] GET /identity/v3/users/2ff9edade73940bf8b6ee80d26abe8c6 => generated 285 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:29.955815 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040e508-b05d-4a99-8790-7fff9b5bd630 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:29.956646 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e87b67-4cf5-4a15-9305-e7c9b0a7ec75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:29.957282 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e87b67-4cf5-4a15-9305-e7c9b0a7ec75 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:29.957562 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e87b67-4cf5-4a15-9305-e7c9b0a7ec75 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:29.957832 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e87b67-4cf5-4a15-9305-e7c9b0a7ec75 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:29.965183 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e87b67-4cf5-4a15-9305-e7c9b0a7ec75 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:29.966664 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e87b67-4cf5-4a15-9305-e7c9b0a7ec75 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:29.978090 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-a4e87b67-4cf5-4a15-9305-e7c9b0a7ec75 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:25:29.978090 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:29.978090 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:29.978090 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:29.978090 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:29.978090 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:29.978090 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:29.978090 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:29.978090 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:29.978090 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:29.978090 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:29.978090 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:29.978090 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:25:29.978090 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:25:29.978090 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:25:29.978090 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:25:29.979408 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:29.979408 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:29.979408 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:25:29.979408 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:25:29.979408 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:25:29.979408 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:25:29.979408 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:29.979408 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:29.979408 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:29.979408 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:29.979408 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:29.979408 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:29.979408 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:29.979408 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:29.979408 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:29.979408 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:29.979408 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:29.979408 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:29.980681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:29.980681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:29.980681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:25:29.980681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:25:29.980681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:25:29.980681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:25:29.980681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:25:29.980681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:25:29.980681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:25:29.980681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:25:29.980681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 528/1074] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Aug 24 05:25:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:25:29.980681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e87b67-4cf5-4a15-9305-e7c9b0a7ec75 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:29.989532 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:30.043979 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:30.061598 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040e508-b05d-4a99-8790-7fff9b5bd630 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:30.085422 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e87b67-4cf5-4a15-9305-e7c9b0a7ec75 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:30.097047 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35361a0-749d-433b-b65b-d5b626e66dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:30.097884 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35361a0-749d-433b-b65b-d5b626e66dbe None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:30.098169 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35361a0-749d-433b-b65b-d5b626e66dbe None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:30.098451 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35361a0-749d-433b-b65b-d5b626e66dbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:30.108628 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35361a0-749d-433b-b65b-d5b626e66dbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:30.110076 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35361a0-749d-433b-b65b-d5b626e66dbe None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:30.119374 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f29bb03-673a-4b73-9516-a3a75ef2a2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:30.120006 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29bb03-673a-4b73-9516-a3a75ef2a2fc None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:30.120292 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29bb03-673a-4b73-9516-a3a75ef2a2fc None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:30.120604 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29bb03-673a-4b73-9516-a3a75ef2a2fc None admin] PATH_INFO: `/v3/projects/42f99470856a442cad8caccfa62dff6a` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:30.122589 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f29bb03-673a-4b73-9516-a3a75ef2a2fc None admin] RBAC: Authorizing `identity:get_project(project_id=42f99470856a442cad8caccfa62dff6a)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:30.124165 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f29bb03-673a-4b73-9516-a3a75ef2a2fc None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:30.124787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-d35361a0-749d-433b-b65b-d5b626e66dbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:25:30.124787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:30.124787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:30.124787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:30.124787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:30.124787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:30.124787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:30.124787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:30.124787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:30.124787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:30.124787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:30.124787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:30.124787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:25:30.124787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:25:30.124787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:25:30.124787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:25:30.125720 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:30.125720 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:30.125720 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:30.125720 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:30.125720 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:30.125720 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:30.125720 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:30.125720 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:30.125720 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:30.125720 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:30.125720 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:30.125720 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:30.125720 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:30.125720 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:30.125720 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:30.125720 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:30.125720 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:25:30.131540 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:25:30.131540 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:25:30.131540 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:30.131713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 529/1075] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:25:29 2022] GET /identity/v3/projects/42f99470856a442cad8caccfa62dff6a => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:30.131785 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 547/1076] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:25:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:25:30.131866 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f29bb03-673a-4b73-9516-a3a75ef2a2fc None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:30.132366 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35361a0-749d-433b-b65b-d5b626e66dbe None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:30.224936 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:30.239314 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f29bb03-673a-4b73-9516-a3a75ef2a2fc None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:30.242483 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35361a0-749d-433b-b65b-d5b626e66dbe None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:30.268392 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:30.276278 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d41af-35dd-4434-b880-4fd85e4c3987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:30.277092 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d41af-35dd-4434-b880-4fd85e4c3987 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:30.277370 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d41af-35dd-4434-b880-4fd85e4c3987 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:30.277647 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d41af-35dd-4434-b880-4fd85e4c3987 None admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:30.277772 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04750f-4e13-4489-ab05-d50afcdc1564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:30.278409 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d41af-35dd-4434-b880-4fd85e4c3987 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:30.278716 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d41af-35dd-4434-b880-4fd85e4c3987 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:30.278994 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04750f-4e13-4489-ab05-d50afcdc1564 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:30.279286 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04750f-4e13-4489-ab05-d50afcdc1564 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:30.279567 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04750f-4e13-4489-ab05-d50afcdc1564 None admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:30.280134 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d41af-35dd-4434-b880-4fd85e4c3987 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:30.280252 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f04750f-4e13-4489-ab05-d50afcdc1564 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:30.280547 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f04750f-4e13-4489-ab05-d50afcdc1564 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:30.282169 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f04750f-4e13-4489-ab05-d50afcdc1564 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:30.294932 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 548/1077] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:25:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:25:30.295626 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04750f-4e13-4489-ab05-d50afcdc1564 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:30.303022 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 530/1078] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Aug 24 05:25:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:30.304228 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d41af-35dd-4434-b880-4fd85e4c3987 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:30.422239 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d41af-35dd-4434-b880-4fd85e4c3987 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:30.426631 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04750f-4e13-4489-ab05-d50afcdc1564 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:30.463879 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27800d78-171a-468c-8326-4e2257c27014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:30.464555 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27800d78-171a-468c-8326-4e2257c27014 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:30.464840 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27800d78-171a-468c-8326-4e2257c27014 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:30.465118 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27800d78-171a-468c-8326-4e2257c27014 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:30.467580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ed0c6d-b9e0-43ac-b150-a5ade0250658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:30.468244 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed0c6d-b9e0-43ac-b150-a5ade0250658 None admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:30.468565 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed0c6d-b9e0-43ac-b150-a5ade0250658 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:30.468845 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed0c6d-b9e0-43ac-b150-a5ade0250658 None admin] PATH_INFO: `/v3/projects/42f99470856a442cad8caccfa62dff6a/users/2ff9edade73940bf8b6ee80d26abe8c6/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:30.471060 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ed0c6d-b9e0-43ac-b150-a5ade0250658 None admin] RBAC: Authorizing `identity:create_grant(project_id=42f99470856a442cad8caccfa62dff6a, user_id=2ff9edade73940bf8b6ee80d26abe8c6, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:30.473805 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ed0c6d-b9e0-43ac-b150-a5ade0250658 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:30.478450 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27800d78-171a-468c-8326-4e2257c27014 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:30.479899 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27800d78-171a-468c-8326-4e2257c27014 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:30.496117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-27800d78-171a-468c-8326-4e2257c27014 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:25:30.496117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:30.496117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:30.496117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:30.496117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:30.496117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:30.496117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:30.496117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:30.496117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:30.496117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:30.496117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:30.496117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:30.496117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:25:30.496117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:25:30.496117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:25:30.496117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:25:30.497819 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:30.497819 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:30.497819 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:30.497819 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:30.497819 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:30.497819 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:30.497819 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:30.497819 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:30.497819 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:30.497819 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:30.497819 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:30.497819 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:30.497819 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:30.497819 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:30.497819 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:30.497819 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:30.497819 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:25:30.498750 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:25:30.498750 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:25:30.498750 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:25:30.498921 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 531/1079] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:25:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:25:30.504924 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27800d78-171a-468c-8326-4e2257c27014 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:30.546201 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:30.556118 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 549/1080] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Aug 24 05:25:30 2022] PUT /identity/v3/projects/42f99470856a442cad8caccfa62dff6a/users/2ff9edade73940bf8b6ee80d26abe8c6/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:25:30.620230 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27800d78-171a-468c-8326-4e2257c27014 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:30.660282 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0fd917-f652-496a-b352-86bf4d042026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:30.660926 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0fd917-f652-496a-b352-86bf4d042026 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:30.661205 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0fd917-f652-496a-b352-86bf4d042026 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:30.661478 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0fd917-f652-496a-b352-86bf4d042026 None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:30.662126 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0fd917-f652-496a-b352-86bf4d042026 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:30.662426 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0fd917-f652-496a-b352-86bf4d042026 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:30.663890 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0fd917-f652-496a-b352-86bf4d042026 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:30.681899 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 532/1081] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:25:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:25:30.683187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ed0c6d-b9e0-43ac-b150-a5ade0250658 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:30.686642 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:30.789377 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ed0c6d-b9e0-43ac-b150-a5ade0250658 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:30.822644 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8527429b-dd60-472d-869e-84afb73fa0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:30.823195 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8527429b-dd60-472d-869e-84afb73fa0de None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:30.823479 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8527429b-dd60-472d-869e-84afb73fa0de None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:30.823760 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8527429b-dd60-472d-869e-84afb73fa0de None admin] PATH_INFO: `/v3/projects/service` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:30.831117 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8527429b-dd60-472d-869e-84afb73fa0de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:30.832610 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8527429b-dd60-472d-869e-84afb73fa0de None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:30.845256 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-8527429b-dd60-472d-869e-84afb73fa0de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:25:30.845256 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:30.845256 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:30.845256 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:30.845256 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:30.845256 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:30.845256 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:30.845256 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:30.845256 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:30.845256 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:30.845256 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:30.845256 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:30.845256 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:25:30.845256 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:25:30.845256 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:25:30.846395 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:25:30.846395 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:30.846395 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:30.846395 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:30.846395 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:30.846395 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:30.846395 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:30.846395 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:30.846395 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:30.846395 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:30.846395 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:30.846395 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:30.846395 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:30.846395 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:30.846395 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:30.846395 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:30.846395 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:30.847297 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:25:30.847297 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:25:30.847297 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:25:30.847297 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:25:30.847297 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:25:30.847297 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:25:30.847297 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:25:30.847297 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:30.847297 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 550/1082] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:25:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:25:30.852301 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0fd917-f652-496a-b352-86bf4d042026 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:30.947863 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:30.951403 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0fd917-f652-496a-b352-86bf4d042026 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:30.985407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee833ef-368f-4ff6-b34e-67e5a07a834c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:30.986011 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee833ef-368f-4ff6-b34e-67e5a07a834c None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:30.986287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee833ef-368f-4ff6-b34e-67e5a07a834c None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:30.986558 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee833ef-368f-4ff6-b34e-67e5a07a834c None admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:30.987248 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee833ef-368f-4ff6-b34e-67e5a07a834c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:30.987552 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee833ef-368f-4ff6-b34e-67e5a07a834c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:30.989006 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee833ef-368f-4ff6-b34e-67e5a07a834c None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:31.000631 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 533/1083] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Aug 24 05:25:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:31.006290 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8527429b-dd60-472d-869e-84afb73fa0de None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:31.067051 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:31.105071 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8527429b-dd60-472d-869e-84afb73fa0de None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:31.138483 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7872c2e2-f8c5-40c9-9f99-1d2341967aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:31.139128 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872c2e2-f8c5-40c9-9f99-1d2341967aad None admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:31.139406 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872c2e2-f8c5-40c9-9f99-1d2341967aad None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:31.139675 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872c2e2-f8c5-40c9-9f99-1d2341967aad None admin] PATH_INFO: `/v3/projects/59592706ae9442e08353158c625e9581/users/fbecffb800584c5786813accd061421b/roles/719a48d7876b4ffa9489dcc5d87cfbad` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:31.141817 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7872c2e2-f8c5-40c9-9f99-1d2341967aad None admin] RBAC: Authorizing `identity:create_grant(project_id=59592706ae9442e08353158c625e9581, user_id=fbecffb800584c5786813accd061421b, role_id=719a48d7876b4ffa9489dcc5d87cfbad)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:31.143243 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7872c2e2-f8c5-40c9-9f99-1d2341967aad None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:31.241395 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 551/1084] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Aug 24 05:25:31 2022] PUT /identity/v3/projects/59592706ae9442e08353158c625e9581/users/fbecffb800584c5786813accd061421b/roles/719a48d7876b4ffa9489dcc5d87cfbad => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:25:32.584680 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85652f4d-d5c0-4255-a049-d272161cb0dc None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:32.585193 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85652f4d-d5c0-4255-a049-d272161cb0dc None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:32.585193 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85652f4d-d5c0-4255-a049-d272161cb0dc None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:32.586481 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 534/1085] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:25:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:25:32.592024 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c85ce-84ca-430a-836f-95475b91480e None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:32.592466 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c85ce-84ca-430a-836f-95475b91480e None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:32.592627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c85ce-84ca-430a-836f-95475b91480e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:32.612415 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-b06c85ce-84ca-430a-836f-95475b91480e None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:25:32.613661 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-b06c85ce-84ca-430a-836f-95475b91480e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:25:32.681728 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 552/1086] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:25:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:25:32.954179 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85652f4d-d5c0-4255-a049-d272161cb0dc None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:33.070595 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85652f4d-d5c0-4255-a049-d272161cb0dc None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:33.108212 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef089c41-5d8a-45d6-a3aa-d80f3c2fee80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:33.108869 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef089c41-5d8a-45d6-a3aa-d80f3c2fee80 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:33.109153 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef089c41-5d8a-45d6-a3aa-d80f3c2fee80 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:33.109434 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef089c41-5d8a-45d6-a3aa-d80f3c2fee80 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:33.119712 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef089c41-5d8a-45d6-a3aa-d80f3c2fee80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:25:33.125128 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-ef089c41-5d8a-45d6-a3aa-d80f3c2fee80 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:25:33.125128 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:33.125128 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:33.125128 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:33.125128 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:33.125128 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:33.125128 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:33.125128 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:33.125128 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:33.125128 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:33.125128 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:33.125128 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:33.125128 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:25:33.125128 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:25:33.125128 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:25:33.125128 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:25:33.126098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:25:33.126098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:25:33.126098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:25:33.126098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:25:33.126098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:33.126098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:33.126098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:33.126098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:33.126098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:33.126098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:33.126098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:33.126098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:33.126098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:33.126098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:33.126098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:33.126098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:33.126953 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:33.126953 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:33.126953 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:33.126953 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:33.126953 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:25:33.126953 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:25:33.126953 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:25:33.126953 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:25:33.126953 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:25:33.126953 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:25:33.126953 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:25:33.126953 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:25:33.126953 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 535/1087] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Aug 24 05:25:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:25:33.132707 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c85ce-84ca-430a-836f-95475b91480e None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:33.188128 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:33.229363 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d373a20-f9df-4493-bfbd-b02bbb9fe980 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:33.229711 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d373a20-f9df-4493-bfbd-b02bbb9fe980 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:33.229997 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d373a20-f9df-4493-bfbd-b02bbb9fe980 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:33.231464 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 536/1088] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:25:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:25:33.235400 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c85ce-84ca-430a-836f-95475b91480e None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:33.236506 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b75c69-3414-4a5e-9e31-3913619a2388 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:33.236795 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b75c69-3414-4a5e-9e31-3913619a2388 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:33.237073 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b75c69-3414-4a5e-9e31-3913619a2388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:33.255292 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-12b75c69-3414-4a5e-9e31-3913619a2388 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:25:33.256686 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-12b75c69-3414-4a5e-9e31-3913619a2388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:25:33.271109 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:33.272247 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e8d2e7-3631-4290-89b0-24a2e047f5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:33.272893 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8d2e7-3631-4290-89b0-24a2e047f5d6 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:33.273172 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8d2e7-3631-4290-89b0-24a2e047f5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:33.273440 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8d2e7-3631-4290-89b0-24a2e047f5d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:33.274117 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e8d2e7-3631-4290-89b0-24a2e047f5d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:33.274626 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e8d2e7-3631-4290-89b0-24a2e047f5d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:33.276134 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e8d2e7-3631-4290-89b0-24a2e047f5d6 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:33.286166 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 553/1089] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:25:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:33.291780 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e8d2e7-3631-4290-89b0-24a2e047f5d6 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:33.326213 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 537/1090] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:25:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:25:33.393907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e8d2e7-3631-4290-89b0-24a2e047f5d6 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:33.408167 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:33.427890 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ea44a2-13fa-4a17-afa0-309b0b14bb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:33.428543 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea44a2-13fa-4a17-afa0-309b0b14bb9d None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:33.428822 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea44a2-13fa-4a17-afa0-309b0b14bb9d None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:33.429095 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea44a2-13fa-4a17-afa0-309b0b14bb9d None admin] PATH_INFO: `/v3/users/2ff9edade73940bf8b6ee80d26abe8c6` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:33.430219 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ea44a2-13fa-4a17-afa0-309b0b14bb9d None admin] RBAC: Authorizing `identity:get_user(user_id=2ff9edade73940bf8b6ee80d26abe8c6)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:33.431626 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ea44a2-13fa-4a17-afa0-309b0b14bb9d None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:33.433231 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 554/1091] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:25:33 2022] GET /identity/v3/users/2ff9edade73940bf8b6ee80d26abe8c6 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:25:33.439796 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b75c69-3414-4a5e-9e31-3913619a2388 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:33.547290 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b75c69-3414-4a5e-9e31-3913619a2388 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:33.582496 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26fccee-d301-43d3-8a81-0b2d729848c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:33.583098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26fccee-d301-43d3-8a81-0b2d729848c8 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:33.583385 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26fccee-d301-43d3-8a81-0b2d729848c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:33.583667 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26fccee-d301-43d3-8a81-0b2d729848c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:33.600866 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ea44a2-13fa-4a17-afa0-309b0b14bb9d None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:33.603053 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:33.657372 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26fccee-d301-43d3-8a81-0b2d729848c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:33.658836 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26fccee-d301-43d3-8a81-0b2d729848c8 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:33.704148 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ea44a2-13fa-4a17-afa0-309b0b14bb9d None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:33.737274 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a360a-7dac-40ed-9832-32b868851415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:33.737898 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a360a-7dac-40ed-9832-32b868851415 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:33.738175 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a360a-7dac-40ed-9832-32b868851415 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:33.738449 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a360a-7dac-40ed-9832-32b868851415 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:33.749634 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53a360a-7dac-40ed-9832-32b868851415 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:25:33.754345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-e53a360a-7dac-40ed-9832-32b868851415 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 05:25:33.754345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:33.754345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:33.754345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:33.754345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:33.754345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:33.754345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:33.754345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:33.754345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:33.754345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:33.754345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:33.754345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:33.754345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:25:33.754345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:25:33.754345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:25:33.754345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:25:33.761346 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:25:33.761346 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:25:33.761346 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:25:33.761346 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:25:33.761346 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:33.761346 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:33.761346 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:33.761346 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:33.761346 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:33.761346 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:33.761346 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:33.761346 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:33.761346 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:33.761346 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:33.761346 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:33.761346 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:33.763090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:33.763090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:33.763090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:33.763090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:33.763090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:25:33.763090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:25:33.763090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:25:33.763090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:25:33.763090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:25:33.763090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:25:33.763090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 05:25:33.763090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:33.763090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 555/1092] 10.222.0.15 () {56 vars in 1120 bytes} [Wed Aug 24 05:25:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:25:33.764138 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 538/1093] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:25:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:25:33.764227 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a360a-7dac-40ed-9832-32b868851415 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:33.765286 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26fccee-d301-43d3-8a81-0b2d729848c8 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:33.807226 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:33.816188 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:33.865550 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a360a-7dac-40ed-9832-32b868851415 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:33.865790 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26fccee-d301-43d3-8a81-0b2d729848c8 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:33.900217 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb7136-6086-4e11-a864-065f6fed9004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:33.900866 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60746d-b373-49d7-b327-a6e06f50cb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:33.901001 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb7136-6086-4e11-a864-065f6fed9004 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:33.901156 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb7136-6086-4e11-a864-065f6fed9004 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:33.901436 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb7136-6086-4e11-a864-065f6fed9004 None admin] PATH_INFO: `/v3/projects/42f99470856a442cad8caccfa62dff6a` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:33.901534 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60746d-b373-49d7-b327-a6e06f50cb94 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:33.901743 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60746d-b373-49d7-b327-a6e06f50cb94 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:33.902036 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60746d-b373-49d7-b327-a6e06f50cb94 None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:33.902558 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eb7136-6086-4e11-a864-065f6fed9004 None admin] RBAC: Authorizing `identity:get_project(project_id=42f99470856a442cad8caccfa62dff6a)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:33.902756 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe60746d-b373-49d7-b327-a6e06f50cb94 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:33.903065 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe60746d-b373-49d7-b327-a6e06f50cb94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:33.904383 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eb7136-6086-4e11-a864-065f6fed9004 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:33.904500 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe60746d-b373-49d7-b327-a6e06f50cb94 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:33.913561 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 539/1094] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:25:33 2022] GET /identity/v3/projects/42f99470856a442cad8caccfa62dff6a => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:33.914902 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb7136-6086-4e11-a864-065f6fed9004 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:33.920970 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 556/1095] 10.222.0.15 () {56 vars in 1115 bytes} [Wed Aug 24 05:25:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:33.922072 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60746d-b373-49d7-b327-a6e06f50cb94 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:34.018929 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb7136-6086-4e11-a864-065f6fed9004 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:34.022054 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60746d-b373-49d7-b327-a6e06f50cb94 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:34.024365 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:34.037003 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:34.055732 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb08ff-7dd0-4e98-86fd-e2233c1a8986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:34.056338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb08ff-7dd0-4e98-86fd-e2233c1a8986 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:34.056660 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb08ff-7dd0-4e98-86fd-e2233c1a8986 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:34.056938 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb08ff-7dd0-4e98-86fd-e2233c1a8986 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:34.057335 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f0158-06c7-4133-bc5b-1091dc69620a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:34.057707 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bb08ff-7dd0-4e98-86fd-e2233c1a8986 None admin] RBAC: Adding query filter params (role.id=e013ad4d436f4cb0a9bf7debb7d13dc7, scope.project.id=42f99470856a442cad8caccfa62dff6a, user.id=2ff9edade73940bf8b6ee80d26abe8c6) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:34.058017 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bb08ff-7dd0-4e98-86fd-e2233c1a8986 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:34.058301 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f0158-06c7-4133-bc5b-1091dc69620a None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:34.058979 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f0158-06c7-4133-bc5b-1091dc69620a None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:34.059756 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f0158-06c7-4133-bc5b-1091dc69620a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:34.060493 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bb08ff-7dd0-4e98-86fd-e2233c1a8986 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:34.066501 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 540/1096] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:25:33 2022] GET /identity/v3/role_assignments?user.id=2ff9edade73940bf8b6ee80d26abe8c6&scope.project.id=42f99470856a442cad8caccfa62dff6a&role.id=e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:34.070646 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14f0158-06c7-4133-bc5b-1091dc69620a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:34.072110 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14f0158-06c7-4133-bc5b-1091dc69620a None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:34.089934 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-a14f0158-06c7-4133-bc5b-1091dc69620a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:25:34.089934 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:34.089934 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:34.089934 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:34.089934 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:34.089934 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:34.089934 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:34.089934 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:34.089934 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:34.089934 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:34.089934 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:34.089934 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:34.089934 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:25:34.089934 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:25:34.089934 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:25:34.089934 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:25:34.091684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:34.091684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:34.091684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:34.091684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:34.091684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:34.091684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:34.091684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:34.091684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:34.091684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:34.091684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:34.091684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:34.091684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:34.091684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:34.091684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:34.091684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:34.091684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:34.091684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:25:34.092634 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:25:34.092634 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:25:34.092634 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:34.092634 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 557/1097] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:25:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:25:34.097437 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb08ff-7dd0-4e98-86fd-e2233c1a8986 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:34.209616 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb08ff-7dd0-4e98-86fd-e2233c1a8986 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:34.253244 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb672a5-7640-4480-823f-dac85bdb2a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:34.253857 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb672a5-7640-4480-823f-dac85bdb2a41 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:34.254143 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb672a5-7640-4480-823f-dac85bdb2a41 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:34.254440 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb672a5-7640-4480-823f-dac85bdb2a41 None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:34.255122 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb672a5-7640-4480-823f-dac85bdb2a41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:34.255431 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb672a5-7640-4480-823f-dac85bdb2a41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:34.257469 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb672a5-7640-4480-823f-dac85bdb2a41 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:34.274349 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 541/1098] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:34.281602 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f0158-06c7-4133-bc5b-1091dc69620a None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:34.383844 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f0158-06c7-4133-bc5b-1091dc69620a None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:34.417674 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b432a15-a111-410c-905c-86b8fa9bb42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:34.418406 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b432a15-a111-410c-905c-86b8fa9bb42c None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:34.418898 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b432a15-a111-410c-905c-86b8fa9bb42c None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:34.419254 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b432a15-a111-410c-905c-86b8fa9bb42c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:34.426940 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b432a15-a111-410c-905c-86b8fa9bb42c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:34.428374 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b432a15-a111-410c-905c-86b8fa9bb42c None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:34.439899 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-0b432a15-a111-410c-905c-86b8fa9bb42c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:25:34.439899 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:34.439899 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:34.439899 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:34.439899 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:34.439899 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:34.439899 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:34.439899 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:34.439899 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:34.439899 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:34.439899 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:34.439899 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:34.439899 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:25:34.439899 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:25:34.439899 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:25:34.439899 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:25:34.441198 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:34.441198 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:34.441198 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:25:34.441198 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:25:34.441198 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:25:34.441198 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:25:34.441198 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:34.441198 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:34.441198 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:34.441198 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:34.441198 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:34.441198 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:34.441198 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:34.441198 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:34.441198 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:34.441198 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:34.441198 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:34.441198 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:34.442155 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:34.442155 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:34.442155 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:25:34.442155 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:25:34.442155 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:25:34.442155 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:25:34.442155 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:25:34.442155 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:25:34.442155 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:25:34.442155 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:34.442155 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 558/1099] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Aug 24 05:25:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:25:34.446759 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb672a5-7640-4480-823f-dac85bdb2a41 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:34.488503 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:34.546775 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb672a5-7640-4480-823f-dac85bdb2a41 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:34.582565 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1397902a-c9ac-434b-91e1-0e54165e1db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:34.583169 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397902a-c9ac-434b-91e1-0e54165e1db2 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:34.583449 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397902a-c9ac-434b-91e1-0e54165e1db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:34.583719 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397902a-c9ac-434b-91e1-0e54165e1db2 None admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:34.584466 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1397902a-c9ac-434b-91e1-0e54165e1db2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:34.584773 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1397902a-c9ac-434b-91e1-0e54165e1db2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:34.586246 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1397902a-c9ac-434b-91e1-0e54165e1db2 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:34.604185 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 542/1100] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Aug 24 05:25:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:34.609801 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b432a15-a111-410c-905c-86b8fa9bb42c None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:34.630076 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:34.718375 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b432a15-a111-410c-905c-86b8fa9bb42c None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:34.758658 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4a83e4-ed6f-44ad-a32e-72a5fe6c31c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:34.759167 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a83e4-ed6f-44ad-a32e-72a5fe6c31c0 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:34.759455 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a83e4-ed6f-44ad-a32e-72a5fe6c31c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:34.762765 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a83e4-ed6f-44ad-a32e-72a5fe6c31c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:34.833605 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4a83e4-ed6f-44ad-a32e-72a5fe6c31c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:34.835060 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4a83e4-ed6f-44ad-a32e-72a5fe6c31c0 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:34.851628 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:34.938256 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 559/1101] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:25:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:25:34.943338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1397902a-c9ac-434b-91e1-0e54165e1db2 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:35.045650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:35.049600 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1397902a-c9ac-434b-91e1-0e54165e1db2 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:35.085191 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2812f3c-68b0-4b7d-bad8-f6b0d13cb2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:35.085883 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2812f3c-68b0-4b7d-bad8-f6b0d13cb2ff None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:35.086168 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2812f3c-68b0-4b7d-bad8-f6b0d13cb2ff None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:35.086441 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2812f3c-68b0-4b7d-bad8-f6b0d13cb2ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:35.096559 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2812f3c-68b0-4b7d-bad8-f6b0d13cb2ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:35.098009 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2812f3c-68b0-4b7d-bad8-f6b0d13cb2ff None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:35.114362 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-e2812f3c-68b0-4b7d-bad8-f6b0d13cb2ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:25:35.114362 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:35.114362 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:35.114362 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:35.114362 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:35.114362 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:35.114362 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:35.114362 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:35.114362 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:35.114362 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:35.114362 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:35.114362 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:35.114362 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:25:35.114362 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:25:35.114362 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:25:35.114362 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:25:35.115359 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:35.115359 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:35.115359 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:35.115359 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:35.115359 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:35.115359 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:35.115359 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:35.115359 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:35.115359 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:35.115359 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:35.115359 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:35.115359 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:35.115359 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:35.115359 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:35.115359 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:35.115359 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:35.115359 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:25:35.116248 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:25:35.116248 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:25:35.116248 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:25:35.116248 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 543/1102] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:25:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:25:35.121361 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4a83e4-ed6f-44ad-a32e-72a5fe6c31c0 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:35.228011 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4a83e4-ed6f-44ad-a32e-72a5fe6c31c0 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:35.244583 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:35.263029 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8b0e6-3edc-4dca-a110-5d63b8cc5e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:35.263727 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8b0e6-3edc-4dca-a110-5d63b8cc5e45 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:35.264005 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8b0e6-3edc-4dca-a110-5d63b8cc5e45 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:35.264336 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8b0e6-3edc-4dca-a110-5d63b8cc5e45 None admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:35.265009 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f8b0e6-3edc-4dca-a110-5d63b8cc5e45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:35.265349 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f8b0e6-3edc-4dca-a110-5d63b8cc5e45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:35.266703 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f8b0e6-3edc-4dca-a110-5d63b8cc5e45 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:35.279150 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 560/1103] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:25:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:35.285088 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2812f3c-68b0-4b7d-bad8-f6b0d13cb2ff None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:35.384163 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2812f3c-68b0-4b7d-bad8-f6b0d13cb2ff None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:35.417942 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6133e6-522a-4afd-b94d-8fad7164c7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:35.418706 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6133e6-522a-4afd-b94d-8fad7164c7fc None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:35.418985 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6133e6-522a-4afd-b94d-8fad7164c7fc None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:35.419257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6133e6-522a-4afd-b94d-8fad7164c7fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:35.426070 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6133e6-522a-4afd-b94d-8fad7164c7fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:35.427504 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6133e6-522a-4afd-b94d-8fad7164c7fc None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:35.438861 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-3c6133e6-522a-4afd-b94d-8fad7164c7fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:25:35.438861 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:35.438861 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:35.438861 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:35.438861 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:35.438861 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:35.438861 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:35.438861 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:35.438861 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:35.438861 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:35.438861 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:35.438861 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:35.438861 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:25:35.438861 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:25:35.438861 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:25:35.439845 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:25:35.439845 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:35.439845 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:35.439845 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:35.439845 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:35.439845 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:35.439845 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:35.439845 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:35.439845 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:35.439845 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:35.439845 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:35.439845 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:35.439845 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:35.439845 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:35.439845 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:35.439845 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:35.439845 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:35.441136 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:25:35.441136 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:25:35.441136 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:25:35.441136 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:25:35.441136 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:25:35.441136 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:25:35.441136 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:25:35.441136 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:25:35.441136 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 544/1104] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:25:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:25:35.445778 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8b0e6-3edc-4dca-a110-5d63b8cc5e45 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:35.548954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8b0e6-3edc-4dca-a110-5d63b8cc5e45 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:35.585886 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2585c75d-a472-47eb-ba70-8db9f37a31be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:35.585886 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585c75d-a472-47eb-ba70-8db9f37a31be None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:35.586209 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585c75d-a472-47eb-ba70-8db9f37a31be None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:35.586446 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585c75d-a472-47eb-ba70-8db9f37a31be None admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:35.587183 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2585c75d-a472-47eb-ba70-8db9f37a31be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:35.587497 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2585c75d-a472-47eb-ba70-8db9f37a31be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:35.589323 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2585c75d-a472-47eb-ba70-8db9f37a31be None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:35.607485 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 561/1105] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Aug 24 05:25:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:25:35.613421 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6133e6-522a-4afd-b94d-8fad7164c7fc None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:35.627255 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:35.715211 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6133e6-522a-4afd-b94d-8fad7164c7fc None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:35.748737 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d794ffe8-5f8e-40fb-990e-f2ded47be8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:35.749354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794ffe8-5f8e-40fb-990e-f2ded47be8e9 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:35.749611 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794ffe8-5f8e-40fb-990e-f2ded47be8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:35.749881 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794ffe8-5f8e-40fb-990e-f2ded47be8e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:35.750593 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d794ffe8-5f8e-40fb-990e-f2ded47be8e9 None admin] RBAC: Adding query filter params (role.id=719a48d7876b4ffa9489dcc5d87cfbad, scope.project.id=59592706ae9442e08353158c625e9581, user.id=fbecffb800584c5786813accd061421b) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:35.750903 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d794ffe8-5f8e-40fb-990e-f2ded47be8e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:35.752307 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d794ffe8-5f8e-40fb-990e-f2ded47be8e9 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:35.757907 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 545/1106] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:25:35 2022] GET /identity/v3/role_assignments?user.id=fbecffb800584c5786813accd061421b&scope.project.id=59592706ae9442e08353158c625e9581&role.id=719a48d7876b4ffa9489dcc5d87cfbad => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:36.038524 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73300df5-f01f-433f-823f-5bb66da0e02e None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:36.039039 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73300df5-f01f-433f-823f-5bb66da0e02e None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:36.039039 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73300df5-f01f-433f-823f-5bb66da0e02e None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:36.040334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 562/1107] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:25:36.046867 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166838c-967d-47db-9da1-df75e531b389 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:36.047145 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166838c-967d-47db-9da1-df75e531b389 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:36.047418 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166838c-967d-47db-9da1-df75e531b389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:36.066405 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-b166838c-967d-47db-9da1-df75e531b389 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:25:36.067605 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-b166838c-967d-47db-9da1-df75e531b389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:25:36.152072 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 546/1108] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:25:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:25:36.435716 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73300df5-f01f-433f-823f-5bb66da0e02e None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:36.476115 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:36.548294 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73300df5-f01f-433f-823f-5bb66da0e02e None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:36.583183 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c32db5-c774-4d0a-9b84-b7366c557b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:36.583985 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c32db5-c774-4d0a-9b84-b7366c557b4f None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:36.584272 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c32db5-c774-4d0a-9b84-b7366c557b4f None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:36.584576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c32db5-c774-4d0a-9b84-b7366c557b4f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:36.585700 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c32db5-c774-4d0a-9b84-b7366c557b4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:36.587117 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c32db5-c774-4d0a-9b84-b7366c557b4f None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:36.588820 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 563/1109] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:25:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:25:36.595587 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b166838c-967d-47db-9da1-df75e531b389 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:36.598925 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:36.706332 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b166838c-967d-47db-9da1-df75e531b389 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:36.739907 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba5cae4-282f-481b-83e9-ec785884ca27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:36.740532 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba5cae4-282f-481b-83e9-ec785884ca27 None admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:36.740811 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba5cae4-282f-481b-83e9-ec785884ca27 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:36.741084 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba5cae4-282f-481b-83e9-ec785884ca27 None admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:36.741760 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba5cae4-282f-481b-83e9-ec785884ca27 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:36.743198 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba5cae4-282f-481b-83e9-ec785884ca27 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:36.863926 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 547/1110] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Aug 24 05:25:36 2022] POST /identity/v3/users => generated 286 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:25:39.018944 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560bf3c-936f-49ee-9db9-41c46b6113d4 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:39.019506 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560bf3c-936f-49ee-9db9-41c46b6113d4 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:39.019506 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560bf3c-936f-49ee-9db9-41c46b6113d4 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:39.020917 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 564/1111] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:25:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:25:39.027321 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2fd66-229b-49a6-bb34-ab265b6f0a2c None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:39.028427 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2fd66-229b-49a6-bb34-ab265b6f0a2c None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:39.028570 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2fd66-229b-49a6-bb34-ab265b6f0a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:39.047292 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-53f2fd66-229b-49a6-bb34-ab265b6f0a2c None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:25:39.048503 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-53f2fd66-229b-49a6-bb34-ab265b6f0a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:25:39.102229 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:39.113696 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 548/1112] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:25:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:25:39.381121 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560bf3c-936f-49ee-9db9-41c46b6113d4 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:39.480909 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560bf3c-936f-49ee-9db9-41c46b6113d4 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:39.483187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:39.513567 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a2f72f-9a62-4006-b9c0-63ae968bbc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:39.514180 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2f72f-9a62-4006-b9c0-63ae968bbc5f None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:39.514461 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2f72f-9a62-4006-b9c0-63ae968bbc5f None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:39.514731 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2f72f-9a62-4006-b9c0-63ae968bbc5f None admin] PATH_INFO: `/v3/roles/0cdb52ffa00449399b27dbfcd383f488` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:39.516092 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a2f72f-9a62-4006-b9c0-63ae968bbc5f None admin] RBAC: Authorizing `identity:get_role(role_id=0cdb52ffa00449399b27dbfcd383f488)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:39.517503 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a2f72f-9a62-4006-b9c0-63ae968bbc5f None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:39.519024 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 565/1113] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:25:39 2022] GET /identity/v3/roles/0cdb52ffa00449399b27dbfcd383f488 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:39.524873 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2fd66-229b-49a6-bb34-ab265b6f0a2c None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:39.625718 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2fd66-229b-49a6-bb34-ab265b6f0a2c None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:39.660679 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb4c558-8e1a-4f4f-a394-a61825a40847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:39.661235 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4c558-8e1a-4f4f-a394-a61825a40847 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:39.661515 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4c558-8e1a-4f4f-a394-a61825a40847 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:39.661792 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4c558-8e1a-4f4f-a394-a61825a40847 None admin] PATH_INFO: `/v3/users/2012896724ec4509bf2d3180c287edc1` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:39.674366 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb4c558-8e1a-4f4f-a394-a61825a40847 None admin] RBAC: Authorizing `identity:get_user(user_id=2012896724ec4509bf2d3180c287edc1)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:39.675811 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb4c558-8e1a-4f4f-a394-a61825a40847 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:39.677424 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 549/1114] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:25:39 2022] GET /identity/v3/users/2012896724ec4509bf2d3180c287edc1 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:39.683444 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a2f72f-9a62-4006-b9c0-63ae968bbc5f None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:39.783494 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a2f72f-9a62-4006-b9c0-63ae968bbc5f None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:39.818478 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea8f07c-54d6-4967-82a8-9b583565fffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:39.819083 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8f07c-54d6-4967-82a8-9b583565fffa None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:39.819369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8f07c-54d6-4967-82a8-9b583565fffa None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:39.819648 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8f07c-54d6-4967-82a8-9b583565fffa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:39.949482 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:39.975850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea8f07c-54d6-4967-82a8-9b583565fffa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:39.977341 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea8f07c-54d6-4967-82a8-9b583565fffa None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:40.076021 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 566/1115] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:25:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:25:40.082172 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb4c558-8e1a-4f4f-a394-a61825a40847 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:40.131231 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:40.182495 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb4c558-8e1a-4f4f-a394-a61825a40847 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:40.217701 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ee1a8-7a28-45ea-8d10-7e78664472c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:40.218260 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ee1a8-7a28-45ea-8d10-7e78664472c6 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:40.218544 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ee1a8-7a28-45ea-8d10-7e78664472c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:40.218819 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ee1a8-7a28-45ea-8d10-7e78664472c6 None admin] PATH_INFO: `/v3/projects/42f99470856a442cad8caccfa62dff6a` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:40.219950 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29ee1a8-7a28-45ea-8d10-7e78664472c6 None admin] RBAC: Authorizing `identity:get_project(project_id=42f99470856a442cad8caccfa62dff6a)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:40.221408 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29ee1a8-7a28-45ea-8d10-7e78664472c6 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:40.223093 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 550/1116] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:25:40 2022] GET /identity/v3/projects/42f99470856a442cad8caccfa62dff6a => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:40.229430 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea8f07c-54d6-4967-82a8-9b583565fffa None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:40.300775 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:40.343812 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea8f07c-54d6-4967-82a8-9b583565fffa None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:40.377911 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8e1025-21c7-4cd9-8e2b-79ffc929d43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:40.378574 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e1025-21c7-4cd9-8e2b-79ffc929d43b None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:40.379079 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e1025-21c7-4cd9-8e2b-79ffc929d43b None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:40.379420 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e1025-21c7-4cd9-8e2b-79ffc929d43b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:40.380667 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8e1025-21c7-4cd9-8e2b-79ffc929d43b None admin] RBAC: Adding query filter params (role.id=0cdb52ffa00449399b27dbfcd383f488, scope.project.id=42f99470856a442cad8caccfa62dff6a, user.id=2012896724ec4509bf2d3180c287edc1) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:40.380982 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8e1025-21c7-4cd9-8e2b-79ffc929d43b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:40.382417 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8e1025-21c7-4cd9-8e2b-79ffc929d43b None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:40.387682 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 567/1117] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:25:40 2022] GET /identity/v3/role_assignments?user.id=2012896724ec4509bf2d3180c287edc1&scope.project.id=42f99470856a442cad8caccfa62dff6a&role.id=0cdb52ffa00449399b27dbfcd383f488 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:42.273095 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261f5e9-e164-42f1-a93b-54da9bf84574 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:42.273646 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261f5e9-e164-42f1-a93b-54da9bf84574 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:42.273646 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261f5e9-e164-42f1-a93b-54da9bf84574 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:42.274944 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 551/1118] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:25:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:25:42.280061 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c8630-cd52-4367-827a-18ee3c856dde None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:42.280517 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c8630-cd52-4367-827a-18ee3c856dde None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:42.280642 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c8630-cd52-4367-827a-18ee3c856dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:42.300850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-c55c8630-cd52-4367-827a-18ee3c856dde None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:25:42.302356 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-c55c8630-cd52-4367-827a-18ee3c856dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:25:42.368883 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 568/1119] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:25:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:25:42.639075 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0261f5e9-e164-42f1-a93b-54da9bf84574 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:42.741465 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0261f5e9-e164-42f1-a93b-54da9bf84574 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:42.758064 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:42.775946 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3186fb4a-5b5f-42b2-9375-17542c05f316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:42.776614 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186fb4a-5b5f-42b2-9375-17542c05f316 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:42.776893 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186fb4a-5b5f-42b2-9375-17542c05f316 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:42.777169 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186fb4a-5b5f-42b2-9375-17542c05f316 None admin] PATH_INFO: `/v3/roles/0cdb52ffa00449399b27dbfcd383f488` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:42.778594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3186fb4a-5b5f-42b2-9375-17542c05f316 None admin] RBAC: Authorizing `identity:get_role(role_id=0cdb52ffa00449399b27dbfcd383f488)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:42.779955 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3186fb4a-5b5f-42b2-9375-17542c05f316 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:42.781322 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 552/1120] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:25:42 2022] GET /identity/v3/roles/0cdb52ffa00449399b27dbfcd383f488 => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:42.787650 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c8630-cd52-4367-827a-18ee3c856dde None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:42.850976 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:42.884273 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c8630-cd52-4367-827a-18ee3c856dde None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:42.918006 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39792fa-342b-433d-8a45-e5a0418af864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:42.918849 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39792fa-342b-433d-8a45-e5a0418af864 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:42.919136 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39792fa-342b-433d-8a45-e5a0418af864 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:42.919415 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39792fa-342b-433d-8a45-e5a0418af864 None admin] PATH_INFO: `/v3/users/2012896724ec4509bf2d3180c287edc1` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:42.920597 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39792fa-342b-433d-8a45-e5a0418af864 None admin] RBAC: Authorizing `identity:get_user(user_id=2012896724ec4509bf2d3180c287edc1)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:42.922026 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39792fa-342b-433d-8a45-e5a0418af864 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:42.923602 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 569/1121] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:25:42 2022] GET /identity/v3/users/2012896724ec4509bf2d3180c287edc1 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:42.929425 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3186fb4a-5b5f-42b2-9375-17542c05f316 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:43.027852 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3186fb4a-5b5f-42b2-9375-17542c05f316 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:43.060675 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d411541-064e-4850-84de-e72294e36a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:43.061288 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d411541-064e-4850-84de-e72294e36a2d None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:43.061561 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d411541-064e-4850-84de-e72294e36a2d None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:43.061827 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d411541-064e-4850-84de-e72294e36a2d None admin] PATH_INFO: `/v3/projects/42f99470856a442cad8caccfa62dff6a` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:43.062925 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d411541-064e-4850-84de-e72294e36a2d None admin] RBAC: Authorizing `identity:get_project(project_id=42f99470856a442cad8caccfa62dff6a)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:43.064332 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d411541-064e-4850-84de-e72294e36a2d None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:43.065945 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 553/1122] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:25:42 2022] GET /identity/v3/projects/42f99470856a442cad8caccfa62dff6a => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:43.072238 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39792fa-342b-433d-8a45-e5a0418af864 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:43.176070 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39792fa-342b-433d-8a45-e5a0418af864 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:43.210859 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ec578-ebd3-4e6c-944b-dc417bed3029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:43.211519 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ec578-ebd3-4e6c-944b-dc417bed3029 None admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:43.211804 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ec578-ebd3-4e6c-944b-dc417bed3029 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:43.212082 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ec578-ebd3-4e6c-944b-dc417bed3029 None admin] PATH_INFO: `/v3/projects/42f99470856a442cad8caccfa62dff6a/users/2012896724ec4509bf2d3180c287edc1/roles/0cdb52ffa00449399b27dbfcd383f488` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:43.214266 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ec578-ebd3-4e6c-944b-dc417bed3029 None admin] RBAC: Authorizing `identity:create_grant(project_id=42f99470856a442cad8caccfa62dff6a, user_id=2012896724ec4509bf2d3180c287edc1, role_id=0cdb52ffa00449399b27dbfcd383f488)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:43.216166 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ec578-ebd3-4e6c-944b-dc417bed3029 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:43.218428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:43.366207 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 570/1123] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Aug 24 05:25:43 2022] PUT /identity/v3/projects/42f99470856a442cad8caccfa62dff6a/users/2012896724ec4509bf2d3180c287edc1/roles/0cdb52ffa00449399b27dbfcd383f488 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:25:45.242914 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc03542-3255-45f7-be34-0849c277ab6e None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:45.243439 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc03542-3255-45f7-be34-0849c277ab6e None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:45.243439 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc03542-3255-45f7-be34-0849c277ab6e None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:45.244826 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 554/1124] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:25:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:25:45.250095 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e639bc-6e54-422c-94da-bcac38d4044f None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:45.250501 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e639bc-6e54-422c-94da-bcac38d4044f None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:45.250639 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e639bc-6e54-422c-94da-bcac38d4044f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:45.268921 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-00e639bc-6e54-422c-94da-bcac38d4044f None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:25:45.270099 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-00e639bc-6e54-422c-94da-bcac38d4044f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:25:45.335105 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 571/1125] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:25:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:25:45.602058 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc03542-3255-45f7-be34-0849c277ab6e None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:45.658140 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:45.704070 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc03542-3255-45f7-be34-0849c277ab6e None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:45.738021 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caca8d6-ba08-4592-bbc5-ca1b76a1c0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:45.738641 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caca8d6-ba08-4592-bbc5-ca1b76a1c0be None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:45.738920 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caca8d6-ba08-4592-bbc5-ca1b76a1c0be None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:45.739193 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caca8d6-ba08-4592-bbc5-ca1b76a1c0be None admin] PATH_INFO: `/v3/roles/0cdb52ffa00449399b27dbfcd383f488` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:45.740634 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caca8d6-ba08-4592-bbc5-ca1b76a1c0be None admin] RBAC: Authorizing `identity:get_role(role_id=0cdb52ffa00449399b27dbfcd383f488)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:45.742448 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caca8d6-ba08-4592-bbc5-ca1b76a1c0be None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:45.743837 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 555/1126] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:25:45 2022] GET /identity/v3/roles/0cdb52ffa00449399b27dbfcd383f488 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:45.750696 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e639bc-6e54-422c-94da-bcac38d4044f None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:45.846850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e639bc-6e54-422c-94da-bcac38d4044f None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:45.869324 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:45.880266 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180f7e45-bf06-415c-9c79-827a628481ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:45.884516 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f7e45-bf06-415c-9c79-827a628481ef None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:45.884516 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f7e45-bf06-415c-9c79-827a628481ef None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:45.884516 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f7e45-bf06-415c-9c79-827a628481ef None admin] PATH_INFO: `/v3/users/2012896724ec4509bf2d3180c287edc1` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:45.884516 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180f7e45-bf06-415c-9c79-827a628481ef None admin] RBAC: Authorizing `identity:get_user(user_id=2012896724ec4509bf2d3180c287edc1)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:45.884516 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180f7e45-bf06-415c-9c79-827a628481ef None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:45.885634 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 572/1127] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:25:45 2022] GET /identity/v3/users/2012896724ec4509bf2d3180c287edc1 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:45.892024 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caca8d6-ba08-4592-bbc5-ca1b76a1c0be None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:45.990336 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caca8d6-ba08-4592-bbc5-ca1b76a1c0be None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:46.009776 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:46.024890 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b889f-6766-45ac-bd7c-5a55fddf00ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:46.025502 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b889f-6766-45ac-bd7c-5a55fddf00ce None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:46.025792 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b889f-6766-45ac-bd7c-5a55fddf00ce None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:46.026055 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b889f-6766-45ac-bd7c-5a55fddf00ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:46.098154 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40b889f-6766-45ac-bd7c-5a55fddf00ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:46.099599 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40b889f-6766-45ac-bd7c-5a55fddf00ce None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:46.196534 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 556/1128] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:25:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:25:46.202544 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180f7e45-bf06-415c-9c79-827a628481ef None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:46.304524 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180f7e45-bf06-415c-9c79-827a628481ef None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:46.338031 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba06efe-6c76-4adc-87d6-9bcc543ffdd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:46.338679 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba06efe-6c76-4adc-87d6-9bcc543ffdd5 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:46.338955 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba06efe-6c76-4adc-87d6-9bcc543ffdd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:46.339251 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba06efe-6c76-4adc-87d6-9bcc543ffdd5 None admin] PATH_INFO: `/v3/projects/42f99470856a442cad8caccfa62dff6a` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:46.340557 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba06efe-6c76-4adc-87d6-9bcc543ffdd5 None admin] RBAC: Authorizing `identity:get_project(project_id=42f99470856a442cad8caccfa62dff6a)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:46.342153 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba06efe-6c76-4adc-87d6-9bcc543ffdd5 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:46.344406 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 573/1129] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:25:46 2022] GET /identity/v3/projects/42f99470856a442cad8caccfa62dff6a => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:46.350856 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b889f-6766-45ac-bd7c-5a55fddf00ce None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:46.372644 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:46.450507 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b889f-6766-45ac-bd7c-5a55fddf00ce None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:46.484829 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e9d0b8-9f96-4ea9-b76e-7446711d8ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:46.485430 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9d0b8-9f96-4ea9-b76e-7446711d8ae2 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:46.485717 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9d0b8-9f96-4ea9-b76e-7446711d8ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:46.485995 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9d0b8-9f96-4ea9-b76e-7446711d8ae2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:46.486718 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e9d0b8-9f96-4ea9-b76e-7446711d8ae2 None admin] RBAC: Adding query filter params (role.id=0cdb52ffa00449399b27dbfcd383f488, scope.project.id=42f99470856a442cad8caccfa62dff6a, user.id=2012896724ec4509bf2d3180c287edc1) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:46.487030 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e9d0b8-9f96-4ea9-b76e-7446711d8ae2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:46.488484 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e9d0b8-9f96-4ea9-b76e-7446711d8ae2 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:46.494132 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 557/1130] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:25:46 2022] GET /identity/v3/role_assignments?user.id=2012896724ec4509bf2d3180c287edc1&scope.project.id=42f99470856a442cad8caccfa62dff6a&role.id=0cdb52ffa00449399b27dbfcd383f488 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:48.435746 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad608036-2e31-4483-96a2-5859b9cb753b None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:48.436249 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad608036-2e31-4483-96a2-5859b9cb753b None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:48.436249 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad608036-2e31-4483-96a2-5859b9cb753b None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:48.437559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 574/1131] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:25:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:25:48.442761 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b3a05f-97f0-4d31-9efd-5a9511465aa1 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:48.443161 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b3a05f-97f0-4d31-9efd-5a9511465aa1 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:48.443319 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b3a05f-97f0-4d31-9efd-5a9511465aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:48.462665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-59b3a05f-97f0-4d31-9efd-5a9511465aa1 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:25:48.463857 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-59b3a05f-97f0-4d31-9efd-5a9511465aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:25:48.530486 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 558/1132] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:25:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:25:48.807842 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad608036-2e31-4483-96a2-5859b9cb753b None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:48.869925 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:48.912057 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad608036-2e31-4483-96a2-5859b9cb753b None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:48.945820 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40032c21-05de-4577-9e46-27455274165f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:48.946449 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40032c21-05de-4577-9e46-27455274165f None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:48.946737 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40032c21-05de-4577-9e46-27455274165f None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:48.947018 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40032c21-05de-4577-9e46-27455274165f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:48.948137 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40032c21-05de-4577-9e46-27455274165f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:48.950233 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40032c21-05de-4577-9e46-27455274165f None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:48.951809 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 575/1133] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:25:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:48.957861 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b3a05f-97f0-4d31-9efd-5a9511465aa1 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:48.973327 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:49.056512 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b3a05f-97f0-4d31-9efd-5a9511465aa1 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:49.090533 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e58c2b2-275f-4f9d-b730-612d131cc2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:49.091077 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58c2b2-275f-4f9d-b730-612d131cc2f9 None admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:49.091360 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58c2b2-275f-4f9d-b730-612d131cc2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:49.091639 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58c2b2-275f-4f9d-b730-612d131cc2f9 None admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:49.092295 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e58c2b2-275f-4f9d-b730-612d131cc2f9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:49.093777 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e58c2b2-275f-4f9d-b730-612d131cc2f9 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:49.190671 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 559/1134] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Aug 24 05:25:48 2022] POST /identity/v3/projects => generated 311 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:25:51.142845 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613bc703-8fd7-4847-8cb9-f73aed590712 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:51.143372 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613bc703-8fd7-4847-8cb9-f73aed590712 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:51.143372 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613bc703-8fd7-4847-8cb9-f73aed590712 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:51.144760 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 576/1135] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:25:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:25:51.150059 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537ecbf-cba6-42f0-b8dc-e6e4ab258f7d None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:51.150451 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537ecbf-cba6-42f0-b8dc-e6e4ab258f7d None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:51.150717 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537ecbf-cba6-42f0-b8dc-e6e4ab258f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:51.169965 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-7537ecbf-cba6-42f0-b8dc-e6e4ab258f7d None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:25:51.171141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-7537ecbf-cba6-42f0-b8dc-e6e4ab258f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:25:51.233585 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:51.241287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 560/1136] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:25:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:25:51.522181 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613bc703-8fd7-4847-8cb9-f73aed590712 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:51.625385 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613bc703-8fd7-4847-8cb9-f73aed590712 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:51.651717 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:51.662688 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dccf9fa-5945-4339-a92f-a4a421ff9ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:51.663328 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dccf9fa-5945-4339-a92f-a4a421ff9ac1 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:51.663606 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dccf9fa-5945-4339-a92f-a4a421ff9ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:51.667053 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dccf9fa-5945-4339-a92f-a4a421ff9ac1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:51.667053 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dccf9fa-5945-4339-a92f-a4a421ff9ac1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:51.667053 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dccf9fa-5945-4339-a92f-a4a421ff9ac1 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:51.668960 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 577/1137] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:25:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:51.674793 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537ecbf-cba6-42f0-b8dc-e6e4ab258f7d None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:51.770936 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537ecbf-cba6-42f0-b8dc-e6e4ab258f7d None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:51.809893 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f23cb-1484-4985-80ce-afac0e381d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:51.809893 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f23cb-1484-4985-80ce-afac0e381d17 None admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:51.809893 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f23cb-1484-4985-80ce-afac0e381d17 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:51.809893 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f23cb-1484-4985-80ce-afac0e381d17 None admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:51.810383 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0f23cb-1484-4985-80ce-afac0e381d17 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:51.810383 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0f23cb-1484-4985-80ce-afac0e381d17 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:51.925931 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 561/1138] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Aug 24 05:25:51 2022] POST /identity/v3/users => generated 286 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:25:53.929798 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff832e42-d6dd-46ae-a33d-322c7ee88504 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:53.930365 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff832e42-d6dd-46ae-a33d-322c7ee88504 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:53.930365 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff832e42-d6dd-46ae-a33d-322c7ee88504 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:53.931686 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 578/1139] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:25:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:25:53.938049 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960647a2-5f20-4881-b228-10386eedbf8e None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:53.938430 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960647a2-5f20-4881-b228-10386eedbf8e None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:53.938613 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960647a2-5f20-4881-b228-10386eedbf8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:53.957172 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-960647a2-5f20-4881-b228-10386eedbf8e None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:25:53.958365 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-960647a2-5f20-4881-b228-10386eedbf8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:25:54.004950 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:54.024207 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 562/1140] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:25:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:25:54.290419 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff832e42-d6dd-46ae-a33d-322c7ee88504 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:54.390547 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff832e42-d6dd-46ae-a33d-322c7ee88504 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:54.423476 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a0d494-3b84-4131-ad6f-8a9b25ff2b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:54.424096 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0d494-3b84-4131-ad6f-8a9b25ff2b9a None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:54.424377 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0d494-3b84-4131-ad6f-8a9b25ff2b9a None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:54.424705 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0d494-3b84-4131-ad6f-8a9b25ff2b9a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:54.435378 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a0d494-3b84-4131-ad6f-8a9b25ff2b9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:25:54.440159 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-58a0d494-3b84-4131-ad6f-8a9b25ff2b9a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:25:54.440159 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:54.440159 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:54.440159 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:54.440159 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:54.440159 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:54.440159 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:54.440159 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:54.440159 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:54.440159 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:54.440159 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:54.440159 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:54.440159 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:25:54.440159 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:25:54.440159 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:25:54.440159 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:25:54.443187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:25:54.443187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:25:54.443187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:25:54.443187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:25:54.443187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:54.443187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:54.443187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:54.443187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:54.443187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:54.443187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:54.443187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:54.443187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:54.443187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:54.443187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:54.443187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:54.443187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:54.444237 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:54.444237 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:54.444237 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:54.444237 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:54.444237 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:25:54.444237 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:25:54.444237 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:25:54.444237 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:25:54.444237 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:25:54.444237 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:25:54.444237 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:25:54.444237 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:54.444237 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 579/1141] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Aug 24 05:25:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:25:54.448714 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960647a2-5f20-4881-b228-10386eedbf8e None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:54.546836 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960647a2-5f20-4881-b228-10386eedbf8e None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:54.579966 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a08868-0359-4d19-8a66-842c28b0b795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:54.580589 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a08868-0359-4d19-8a66-842c28b0b795 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:54.580872 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a08868-0359-4d19-8a66-842c28b0b795 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:54.581149 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a08868-0359-4d19-8a66-842c28b0b795 None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:54.581806 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a08868-0359-4d19-8a66-842c28b0b795 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:54.582112 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a08868-0359-4d19-8a66-842c28b0b795 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:54.583475 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a08868-0359-4d19-8a66-842c28b0b795 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:54.591375 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 563/1142] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:25:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:54.597490 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a0d494-3b84-4131-ad6f-8a9b25ff2b9a None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:54.654575 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:54.700256 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a0d494-3b84-4131-ad6f-8a9b25ff2b9a None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:54.736224 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32cd148-c7f9-4cd2-bb37-364b8cc1ad9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:54.736882 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32cd148-c7f9-4cd2-bb37-364b8cc1ad9d None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:54.737166 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32cd148-c7f9-4cd2-bb37-364b8cc1ad9d None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:54.737442 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32cd148-c7f9-4cd2-bb37-364b8cc1ad9d None admin] PATH_INFO: `/v3/users/128f840ceae14ba088a3517eee4f190b` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:54.750816 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32cd148-c7f9-4cd2-bb37-364b8cc1ad9d None admin] RBAC: Authorizing `identity:get_user(user_id=128f840ceae14ba088a3517eee4f190b)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:54.752288 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32cd148-c7f9-4cd2-bb37-364b8cc1ad9d None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:54.753970 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 580/1143] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:25:54 2022] GET /identity/v3/users/128f840ceae14ba088a3517eee4f190b => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:54.760231 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a08868-0359-4d19-8a66-842c28b0b795 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:54.806305 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:54.860014 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a08868-0359-4d19-8a66-842c28b0b795 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:54.893975 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d188c-1d4e-46fe-8ce6-5576ab0bdb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:54.894574 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d188c-1d4e-46fe-8ce6-5576ab0bdb89 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:54.894856 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d188c-1d4e-46fe-8ce6-5576ab0bdb89 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:54.895129 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d188c-1d4e-46fe-8ce6-5576ab0bdb89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:54.961526 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d188c-1d4e-46fe-8ce6-5576ab0bdb89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:54.962895 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d188c-1d4e-46fe-8ce6-5576ab0bdb89 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:55.005439 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:55.061330 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 564/1144] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:25:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:25:55.067520 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32cd148-c7f9-4cd2-bb37-364b8cc1ad9d None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:55.167103 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32cd148-c7f9-4cd2-bb37-364b8cc1ad9d None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:55.181456 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:55.201414 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6c548-98ce-4146-b049-c2046e32d0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:55.202047 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6c548-98ce-4146-b049-c2046e32d0ff None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:55.202326 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6c548-98ce-4146-b049-c2046e32d0ff None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:55.202601 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6c548-98ce-4146-b049-c2046e32d0ff None admin] PATH_INFO: `/v3/projects/49f6e56e3fff42a59eaa4d34b2c51c9d` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:55.203707 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6c548-98ce-4146-b049-c2046e32d0ff None admin] RBAC: Authorizing `identity:get_project(project_id=49f6e56e3fff42a59eaa4d34b2c51c9d)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:55.205206 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6c548-98ce-4146-b049-c2046e32d0ff None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:55.206829 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 581/1145] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:25:55 2022] GET /identity/v3/projects/49f6e56e3fff42a59eaa4d34b2c51c9d => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:55.213105 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d188c-1d4e-46fe-8ce6-5576ab0bdb89 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:55.312672 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d188c-1d4e-46fe-8ce6-5576ab0bdb89 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:55.352114 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce7d248-050f-4f63-bd62-8f11f1d1dcfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:55.352747 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7d248-050f-4f63-bd62-8f11f1d1dcfb None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:55.353030 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7d248-050f-4f63-bd62-8f11f1d1dcfb None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:55.353409 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7d248-050f-4f63-bd62-8f11f1d1dcfb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:55.354159 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce7d248-050f-4f63-bd62-8f11f1d1dcfb None admin] RBAC: Adding query filter params (role.id=e013ad4d436f4cb0a9bf7debb7d13dc7, scope.project.id=49f6e56e3fff42a59eaa4d34b2c51c9d, user.id=128f840ceae14ba088a3517eee4f190b) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:55.354470 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce7d248-050f-4f63-bd62-8f11f1d1dcfb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:55.356479 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce7d248-050f-4f63-bd62-8f11f1d1dcfb None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:55.361764 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 565/1146] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:25:55 2022] GET /identity/v3/role_assignments?user.id=128f840ceae14ba088a3517eee4f190b&scope.project.id=49f6e56e3fff42a59eaa4d34b2c51c9d&role.id=e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:57.242948 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00331e3-fdb5-44fc-8b0f-751b1f67eda8 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:57.243511 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00331e3-fdb5-44fc-8b0f-751b1f67eda8 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:57.243511 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00331e3-fdb5-44fc-8b0f-751b1f67eda8 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:57.244877 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 582/1147] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:25:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:25:57.250762 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6945b8-8213-4ede-b4a7-1fb85c5e5605 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:57.251197 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6945b8-8213-4ede-b4a7-1fb85c5e5605 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:57.251297 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6945b8-8213-4ede-b4a7-1fb85c5e5605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:57.269450 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-3d6945b8-8213-4ede-b4a7-1fb85c5e5605 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:25:57.270619 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-3d6945b8-8213-4ede-b4a7-1fb85c5e5605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:25:57.272549 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:57.336464 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 566/1148] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:25:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:25:57.605829 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00331e3-fdb5-44fc-8b0f-751b1f67eda8 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:57.722177 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00331e3-fdb5-44fc-8b0f-751b1f67eda8 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:57.767422 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2328b02d-4534-47a1-bb75-fd03d556f9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:57.768094 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328b02d-4534-47a1-bb75-fd03d556f9d5 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:57.768381 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328b02d-4534-47a1-bb75-fd03d556f9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:57.768691 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328b02d-4534-47a1-bb75-fd03d556f9d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:57.781876 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2328b02d-4534-47a1-bb75-fd03d556f9d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:25:57.789538 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-2328b02d-4534-47a1-bb75-fd03d556f9d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:25:57.789538 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:25:57.789538 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:25:57.789538 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:25:57.789538 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:25:57.789538 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:25:57.789538 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:25:57.789538 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:25:57.789538 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:25:57.789538 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:25:57.789538 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:25:57.789538 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:25:57.789538 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:25:57.789538 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:25:57.789538 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:25:57.789538 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:25:57.790717 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:25:57.790717 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:25:57.790717 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:25:57.790717 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:25:57.790717 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:25:57.790717 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:25:57.790717 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:25:57.790717 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:25:57.790717 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:25:57.790717 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:25:57.790717 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:25:57.790717 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:25:57.790717 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:25:57.790717 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:25:57.790717 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:25:57.790717 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:25:57.791989 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:25:57.791989 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:25:57.791989 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:25:57.791989 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:25:57.791989 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:25:57.791989 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:25:57.791989 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:25:57.791989 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:25:57.791989 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:25:57.791989 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:25:57.791989 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:25:57.791989 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:25:57.791989 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 583/1149] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Aug 24 05:25:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:25:57.797680 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6945b8-8213-4ede-b4a7-1fb85c5e5605 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:57.895160 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6945b8-8213-4ede-b4a7-1fb85c5e5605 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:57.930123 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0922cfd1-0c2d-48c1-aded-9c0a7465ec09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:57.930719 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922cfd1-0c2d-48c1-aded-9c0a7465ec09 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:57.931003 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922cfd1-0c2d-48c1-aded-9c0a7465ec09 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:57.931277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922cfd1-0c2d-48c1-aded-9c0a7465ec09 None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:57.932012 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0922cfd1-0c2d-48c1-aded-9c0a7465ec09 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:25:57.932319 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0922cfd1-0c2d-48c1-aded-9c0a7465ec09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:57.933826 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0922cfd1-0c2d-48c1-aded-9c0a7465ec09 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:57.938748 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:57.942004 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 567/1150] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:25:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:57.947966 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2328b02d-4534-47a1-bb75-fd03d556f9d5 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:57.991263 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:58.046062 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2328b02d-4534-47a1-bb75-fd03d556f9d5 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:58.087279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb1870b-930b-4fb5-9dfe-304197b05468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:58.087904 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1870b-930b-4fb5-9dfe-304197b05468 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:58.088181 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1870b-930b-4fb5-9dfe-304197b05468 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:58.088480 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1870b-930b-4fb5-9dfe-304197b05468 None admin] PATH_INFO: `/v3/users/128f840ceae14ba088a3517eee4f190b` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:58.090241 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb1870b-930b-4fb5-9dfe-304197b05468 None admin] RBAC: Authorizing `identity:get_user(user_id=128f840ceae14ba088a3517eee4f190b)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:58.091810 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb1870b-930b-4fb5-9dfe-304197b05468 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:58.093424 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 584/1151] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:25:57 2022] GET /identity/v3/users/128f840ceae14ba088a3517eee4f190b => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:58.099233 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0922cfd1-0c2d-48c1-aded-9c0a7465ec09 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:58.197898 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0922cfd1-0c2d-48c1-aded-9c0a7465ec09 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:58.231801 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd44b275-75ec-405f-9cc5-8bf294fe1d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:58.232433 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd44b275-75ec-405f-9cc5-8bf294fe1d1b None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:58.232724 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd44b275-75ec-405f-9cc5-8bf294fe1d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:58.232983 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd44b275-75ec-405f-9cc5-8bf294fe1d1b None admin] PATH_INFO: `/v3/projects/49f6e56e3fff42a59eaa4d34b2c51c9d` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:58.234082 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd44b275-75ec-405f-9cc5-8bf294fe1d1b None admin] RBAC: Authorizing `identity:get_project(project_id=49f6e56e3fff42a59eaa4d34b2c51c9d)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:58.235491 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd44b275-75ec-405f-9cc5-8bf294fe1d1b None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:58.237116 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 568/1152] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:25:58 2022] GET /identity/v3/projects/49f6e56e3fff42a59eaa4d34b2c51c9d => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:25:58.243202 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb1870b-930b-4fb5-9dfe-304197b05468 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:25:58.342417 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb1870b-930b-4fb5-9dfe-304197b05468 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:25:58.352428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:25:58.375267 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1255db-82e4-4d3e-855f-ec9f6db0039e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:25:58.375922 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1255db-82e4-4d3e-855f-ec9f6db0039e None admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:25:58.376206 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1255db-82e4-4d3e-855f-ec9f6db0039e None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:25:58.376513 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1255db-82e4-4d3e-855f-ec9f6db0039e None admin] PATH_INFO: `/v3/projects/49f6e56e3fff42a59eaa4d34b2c51c9d/users/128f840ceae14ba088a3517eee4f190b/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:25:58.378428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1255db-82e4-4d3e-855f-ec9f6db0039e None admin] RBAC: Authorizing `identity:create_grant(project_id=49f6e56e3fff42a59eaa4d34b2c51c9d, user_id=128f840ceae14ba088a3517eee4f190b, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:25:58.379863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1255db-82e4-4d3e-855f-ec9f6db0039e None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:25:58.496212 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 585/1153] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Aug 24 05:25:58 2022] PUT /identity/v3/projects/49f6e56e3fff42a59eaa4d34b2c51c9d/users/128f840ceae14ba088a3517eee4f190b/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:26:00.376760 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558919b2-4873-4d65-888d-0406efa33ab0 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:00.377312 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558919b2-4873-4d65-888d-0406efa33ab0 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:00.377312 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558919b2-4873-4d65-888d-0406efa33ab0 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:00.378643 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 569/1154] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:26:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:26:00.383893 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827fa9d9-b7c8-46f7-84b9-f25a3e459376 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:00.384324 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827fa9d9-b7c8-46f7-84b9-f25a3e459376 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:00.384462 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827fa9d9-b7c8-46f7-84b9-f25a3e459376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:00.402885 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-827fa9d9-b7c8-46f7-84b9-f25a3e459376 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:26:00.404071 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-827fa9d9-b7c8-46f7-84b9-f25a3e459376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:26:00.470050 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 586/1155] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:26:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:26:00.743029 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558919b2-4873-4d65-888d-0406efa33ab0 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:26:00.811325 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:26:00.846480 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558919b2-4873-4d65-888d-0406efa33ab0 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:26:00.880627 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f4d2a-a403-494d-8120-a83f95394ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:26:00.881247 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f4d2a-a403-494d-8120-a83f95394ab1 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:00.881527 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f4d2a-a403-494d-8120-a83f95394ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:00.881797 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f4d2a-a403-494d-8120-a83f95394ab1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:00.892184 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f4d2a-a403-494d-8120-a83f95394ab1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:26:00.896833 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-ba4f4d2a-a403-494d-8120-a83f95394ab1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:26:00.896833 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:26:00.896833 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:26:00.896833 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:26:00.896833 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:26:00.896833 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:26:00.896833 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:26:00.896833 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:26:00.896833 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:26:00.896833 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:26:00.896833 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:26:00.896833 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:26:00.896833 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:26:00.896833 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:26:00.896833 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:26:00.896833 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:26:00.898011 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:26:00.898011 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:26:00.898011 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:26:00.898011 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:26:00.898011 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:26:00.898011 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:26:00.898011 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:26:00.898011 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:26:00.898011 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:26:00.898011 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:26:00.898011 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:26:00.898011 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:26:00.898011 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:26:00.898011 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:26:00.898011 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:26:00.898011 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:26:00.899061 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:26:00.899061 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:26:00.899061 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:26:00.899061 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:26:00.899061 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:26:00.899061 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:26:00.899061 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:26:00.899061 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:26:00.899061 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:26:00.899061 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:26:00.899061 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:26:00.899061 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:26:00.899061 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 570/1156] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Aug 24 05:26:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:26:00.904712 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827fa9d9-b7c8-46f7-84b9-f25a3e459376 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:26:00.998396 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:26:01.001190 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827fa9d9-b7c8-46f7-84b9-f25a3e459376 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:26:01.034772 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a87ef-cdd1-431b-9ff3-871bd7a9a9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:26:01.035374 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a87ef-cdd1-431b-9ff3-871bd7a9a9cc None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:01.035651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a87ef-cdd1-431b-9ff3-871bd7a9a9cc None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:01.035927 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a87ef-cdd1-431b-9ff3-871bd7a9a9cc None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:01.036615 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9a87ef-cdd1-431b-9ff3-871bd7a9a9cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:26:01.036919 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9a87ef-cdd1-431b-9ff3-871bd7a9a9cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:26:01.038350 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9a87ef-cdd1-431b-9ff3-871bd7a9a9cc None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:26:01.046631 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 587/1157] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:26:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:26:01.052549 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f4d2a-a403-494d-8120-a83f95394ab1 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:26:01.149520 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f4d2a-a403-494d-8120-a83f95394ab1 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:26:01.175875 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:26:01.184106 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43da6001-3f61-4e21-ae38-f2a456bf2174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:26:01.187904 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da6001-3f61-4e21-ae38-f2a456bf2174 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:01.187904 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da6001-3f61-4e21-ae38-f2a456bf2174 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:01.187904 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da6001-3f61-4e21-ae38-f2a456bf2174 None admin] PATH_INFO: `/v3/users/128f840ceae14ba088a3517eee4f190b` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:01.187904 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43da6001-3f61-4e21-ae38-f2a456bf2174 None admin] RBAC: Authorizing `identity:get_user(user_id=128f840ceae14ba088a3517eee4f190b)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:26:01.188806 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43da6001-3f61-4e21-ae38-f2a456bf2174 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:26:01.190179 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 571/1158] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:26:01 2022] GET /identity/v3/users/128f840ceae14ba088a3517eee4f190b => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:26:01.196149 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a87ef-cdd1-431b-9ff3-871bd7a9a9cc None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:26:01.295371 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a87ef-cdd1-431b-9ff3-871bd7a9a9cc None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:26:01.329590 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b06773-6663-4fa0-8efe-3662e322da0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:26:01.330197 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b06773-6663-4fa0-8efe-3662e322da0f None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:01.330481 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b06773-6663-4fa0-8efe-3662e322da0f None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:01.330757 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b06773-6663-4fa0-8efe-3662e322da0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:01.360366 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:26:01.396980 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b06773-6663-4fa0-8efe-3662e322da0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:26:01.398511 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b06773-6663-4fa0-8efe-3662e322da0f None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:26:01.505600 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 588/1159] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:26:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:26:01.511971 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43da6001-3f61-4e21-ae38-f2a456bf2174 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:26:01.610049 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43da6001-3f61-4e21-ae38-f2a456bf2174 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:26:01.643898 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d8ef67-123e-4800-903f-19166a9a72bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:26:01.644564 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d8ef67-123e-4800-903f-19166a9a72bf None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:01.644859 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d8ef67-123e-4800-903f-19166a9a72bf None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:01.645141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d8ef67-123e-4800-903f-19166a9a72bf None admin] PATH_INFO: `/v3/projects/49f6e56e3fff42a59eaa4d34b2c51c9d` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:01.646271 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d8ef67-123e-4800-903f-19166a9a72bf None admin] RBAC: Authorizing `identity:get_project(project_id=49f6e56e3fff42a59eaa4d34b2c51c9d)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:26:01.647714 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d8ef67-123e-4800-903f-19166a9a72bf None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:26:01.649490 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 572/1160] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:26:01 2022] GET /identity/v3/projects/49f6e56e3fff42a59eaa4d34b2c51c9d => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:26:01.656880 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b06773-6663-4fa0-8efe-3662e322da0f None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:26:01.719294 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:26:01.766229 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b06773-6663-4fa0-8efe-3662e322da0f None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:26:01.798833 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cdfbaf-c41c-455d-a646-3ed0fa1e4be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:26:01.799432 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cdfbaf-c41c-455d-a646-3ed0fa1e4be2 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:01.799710 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cdfbaf-c41c-455d-a646-3ed0fa1e4be2 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:01.799983 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cdfbaf-c41c-455d-a646-3ed0fa1e4be2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:01.800725 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cdfbaf-c41c-455d-a646-3ed0fa1e4be2 None admin] RBAC: Adding query filter params (role.id=e013ad4d436f4cb0a9bf7debb7d13dc7, scope.project.id=49f6e56e3fff42a59eaa4d34b2c51c9d, user.id=128f840ceae14ba088a3517eee4f190b) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:26:01.801033 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cdfbaf-c41c-455d-a646-3ed0fa1e4be2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:26:01.802442 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cdfbaf-c41c-455d-a646-3ed0fa1e4be2 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:26:01.808865 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 589/1161] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:26:01 2022] GET /identity/v3/role_assignments?user.id=128f840ceae14ba088a3517eee4f190b&scope.project.id=49f6e56e3fff42a59eaa4d34b2c51c9d&role.id=e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:26:03.746870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a31849-d72e-4b8d-8f10-cffa8aa31584 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:03.747439 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a31849-d72e-4b8d-8f10-cffa8aa31584 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:03.747583 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a31849-d72e-4b8d-8f10-cffa8aa31584 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:03.749740 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 573/1162] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:26:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:26:03.756162 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82958a5a-5447-4e18-92b2-6b09ae43dac9 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:03.756635 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82958a5a-5447-4e18-92b2-6b09ae43dac9 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:03.756730 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82958a5a-5447-4e18-92b2-6b09ae43dac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:03.776291 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-82958a5a-5447-4e18-92b2-6b09ae43dac9 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:26:03.777537 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-82958a5a-5447-4e18-92b2-6b09ae43dac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:26:03.843709 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 590/1163] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:26:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:26:04.111958 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a31849-d72e-4b8d-8f10-cffa8aa31584 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:26:04.176049 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:26:04.215828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a31849-d72e-4b8d-8f10-cffa8aa31584 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:26:04.250695 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdfe774-86f7-4add-b8fb-f01d0f59cd9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:26:04.251292 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfe774-86f7-4add-b8fb-f01d0f59cd9c None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:04.251574 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfe774-86f7-4add-b8fb-f01d0f59cd9c None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:04.251845 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfe774-86f7-4add-b8fb-f01d0f59cd9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:04.317583 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdfe774-86f7-4add-b8fb-f01d0f59cd9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:26:04.318935 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdfe774-86f7-4add-b8fb-f01d0f59cd9c None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:26:04.378372 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:26:04.415475 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 574/1164] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:26:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:26:04.421616 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82958a5a-5447-4e18-92b2-6b09ae43dac9 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:26:04.475693 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:26:04.518224 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82958a5a-5447-4e18-92b2-6b09ae43dac9 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:26:04.550858 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf06d3b6-c1f0-4086-8a54-ade5c2e71d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:26:04.551471 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf06d3b6-c1f0-4086-8a54-ade5c2e71d95 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:04.551747 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf06d3b6-c1f0-4086-8a54-ade5c2e71d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:04.552020 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf06d3b6-c1f0-4086-8a54-ade5c2e71d95 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:04.559612 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf06d3b6-c1f0-4086-8a54-ade5c2e71d95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:26:04.561169 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf06d3b6-c1f0-4086-8a54-ade5c2e71d95 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:26:04.572358 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-bf06d3b6-c1f0-4086-8a54-ade5c2e71d95 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 05:26:04.572358 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:26:04.572358 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 24 05:26:04.572358 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 24 05:26:04.572358 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:26:04.572358 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:26:04.572358 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:26:04.572358 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:26:04.572358 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 24 05:26:04.572358 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:26:04.572358 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:26:04.572358 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:26:04.572358 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:26:04.572358 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:26:04.572358 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:26:04.572358 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:26:04.572358 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:26:04.574239 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:26:04.574239 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:26:04.574239 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:26:04.574239 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:26:04.574239 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:26:04.574239 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:26:04.574239 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:26:04.574239 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:26:04.574239 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:26:04.574239 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:26:04.574239 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:26:04.574239 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:26:04.574239 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:26:04.574239 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:26:04.574239 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:26:04.574239 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:26:04.575437 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:26:04.575437 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:26:04.575437 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:26:04.575437 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:26:04.575437 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:26:04.575437 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:26:04.575437 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:26:04.575437 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:26:04.575437 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:26:04.575437 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:26:04.575437 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 24 05:26:04.575437 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:26:04.575437 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 05:26:04.575437 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:26:04.575437 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 591/1165] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Aug 24 05:26:04 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:26:04.580229 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdfe774-86f7-4add-b8fb-f01d0f59cd9c None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:26:04.680174 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdfe774-86f7-4add-b8fb-f01d0f59cd9c None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:26:04.714111 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a2188-18d6-4603-b2dc-6728ff5df034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:26:04.714712 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a2188-18d6-4603-b2dc-6728ff5df034 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:04.714992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a2188-18d6-4603-b2dc-6728ff5df034 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:04.715266 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a2188-18d6-4603-b2dc-6728ff5df034 None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:04.715903 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a2188-18d6-4603-b2dc-6728ff5df034 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:26:04.716206 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a2188-18d6-4603-b2dc-6728ff5df034 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:26:04.717611 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a2188-18d6-4603-b2dc-6728ff5df034 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:26:04.724513 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 575/1166] 10.222.0.15 () {56 vars in 1115 bytes} [Wed Aug 24 05:26:04 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:26:06.617889 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8dd3f4-ec1b-491f-b72f-2594bd168a00 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:06.618408 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8dd3f4-ec1b-491f-b72f-2594bd168a00 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:06.618408 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8dd3f4-ec1b-491f-b72f-2594bd168a00 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:06.619696 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 592/1167] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:26:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:26:06.625244 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ccf35d-8385-45fe-a413-9952e2652785 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:06.625779 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ccf35d-8385-45fe-a413-9952e2652785 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:06.625779 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ccf35d-8385-45fe-a413-9952e2652785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:06.644445 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-43ccf35d-8385-45fe-a413-9952e2652785 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:26:06.645655 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-43ccf35d-8385-45fe-a413-9952e2652785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:26:06.669588 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:26:06.719929 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 576/1168] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:26:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:26:06.988743 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8dd3f4-ec1b-491f-b72f-2594bd168a00 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:26:07.087532 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8dd3f4-ec1b-491f-b72f-2594bd168a00 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:26:07.103038 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:26:07.120618 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ac908-08af-4400-9d1b-aa7eea42a191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:26:07.127476 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ac908-08af-4400-9d1b-aa7eea42a191 None admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:07.127476 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ac908-08af-4400-9d1b-aa7eea42a191 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:07.127476 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ac908-08af-4400-9d1b-aa7eea42a191 None admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:07.127476 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29ac908-08af-4400-9d1b-aa7eea42a191 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:26:07.127476 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29ac908-08af-4400-9d1b-aa7eea42a191 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:26:07.258812 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 593/1169] 10.222.0.15 () {60 vars in 1138 bytes} [Wed Aug 24 05:26:06 2022] POST /identity/v3/domains => generated 267 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:26:09.168607 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad87334-7fab-4631-b3a4-7a99e63d30c1 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:09.169142 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad87334-7fab-4631-b3a4-7a99e63d30c1 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:09.169142 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad87334-7fab-4631-b3a4-7a99e63d30c1 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:09.170495 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 577/1170] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:26:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:26:09.175768 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2afca4d-5083-40b0-aa69-387fccf5cafd None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:09.176194 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2afca4d-5083-40b0-aa69-387fccf5cafd None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:09.176333 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2afca4d-5083-40b0-aa69-387fccf5cafd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:09.194418 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-a2afca4d-5083-40b0-aa69-387fccf5cafd None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:26:09.195658 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-a2afca4d-5083-40b0-aa69-387fccf5cafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:26:09.261536 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 594/1171] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:26:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:26:09.533305 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad87334-7fab-4631-b3a4-7a99e63d30c1 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:26:09.637640 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad87334-7fab-4631-b3a4-7a99e63d30c1 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:26:09.672372 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb2821-93f1-48f0-b887-6b218643b811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:26:09.673025 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb2821-93f1-48f0-b887-6b218643b811 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:09.673312 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb2821-93f1-48f0-b887-6b218643b811 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:09.673616 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb2821-93f1-48f0-b887-6b218643b811 None admin] PATH_INFO: `/v3/domains/da73987d325848b49482d01f3b04d6a9` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:09.684587 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcb2821-93f1-48f0-b887-6b218643b811 None admin] RBAC: Authorizing `identity:get_domain(domain_id=da73987d325848b49482d01f3b04d6a9)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:26:09.686030 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcb2821-93f1-48f0-b887-6b218643b811 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:26:09.687621 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 578/1172] 10.222.0.15 () {56 vars in 1183 bytes} [Wed Aug 24 05:26:09 2022] GET /identity/v3/domains/da73987d325848b49482d01f3b04d6a9 => generated 267 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:26:09.693745 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2afca4d-5083-40b0-aa69-387fccf5cafd None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:26:09.790668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2afca4d-5083-40b0-aa69-387fccf5cafd None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:26:09.802879 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:26:09.834195 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e6ea48-bd51-4c48-b172-3a5497cfaef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:26:09.834807 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6ea48-bd51-4c48-b172-3a5497cfaef7 None admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:09.835091 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6ea48-bd51-4c48-b172-3a5497cfaef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:09.835369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6ea48-bd51-4c48-b172-3a5497cfaef7 None admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:09.836026 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e6ea48-bd51-4c48-b172-3a5497cfaef7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:26:09.837552 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e6ea48-bd51-4c48-b172-3a5497cfaef7 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:26:09.950757 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 595/1173] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Aug 24 05:26:09 2022] POST /identity/v3/projects => generated 361 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:26:11.958443 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414966d9-f819-4997-ab90-03bc7fa47912 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:11.958981 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414966d9-f819-4997-ab90-03bc7fa47912 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:11.958981 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414966d9-f819-4997-ab90-03bc7fa47912 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:11.960321 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 579/1174] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:26:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:26:11.965742 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc7492-06c6-4326-95c0-9bf7b49b733d None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:11.966185 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc7492-06c6-4326-95c0-9bf7b49b733d None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:11.966279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc7492-06c6-4326-95c0-9bf7b49b733d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:11.984541 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-57bc7492-06c6-4326-95c0-9bf7b49b733d None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:26:11.985731 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-57bc7492-06c6-4326-95c0-9bf7b49b733d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:26:12.053041 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 596/1175] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:26:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:26:12.320586 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414966d9-f819-4997-ab90-03bc7fa47912 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:26:12.340906 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:26:12.424646 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414966d9-f819-4997-ab90-03bc7fa47912 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:26:12.458922 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c252a9-e821-4ba4-a8a0-3d2decf322ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:26:12.459547 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c252a9-e821-4ba4-a8a0-3d2decf322ad None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:12.459831 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c252a9-e821-4ba4-a8a0-3d2decf322ad None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:12.460109 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c252a9-e821-4ba4-a8a0-3d2decf322ad None admin] PATH_INFO: `/v3/domains/da73987d325848b49482d01f3b04d6a9` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:12.461262 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c252a9-e821-4ba4-a8a0-3d2decf322ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=da73987d325848b49482d01f3b04d6a9)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:26:12.462700 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c252a9-e821-4ba4-a8a0-3d2decf322ad None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:26:12.464900 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 580/1176] 10.222.0.15 () {56 vars in 1183 bytes} [Wed Aug 24 05:26:12 2022] GET /identity/v3/domains/da73987d325848b49482d01f3b04d6a9 => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:26:12.470685 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc7492-06c6-4326-95c0-9bf7b49b733d None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:26:12.554633 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:26:12.565910 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc7492-06c6-4326-95c0-9bf7b49b733d None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:26:12.599253 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878234aa-9cfd-4167-8c28-43a481c05cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:26:12.599778 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878234aa-9cfd-4167-8c28-43a481c05cd5 None admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:12.600056 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878234aa-9cfd-4167-8c28-43a481c05cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:12.600332 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878234aa-9cfd-4167-8c28-43a481c05cd5 None admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:12.601003 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878234aa-9cfd-4167-8c28-43a481c05cd5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:26:12.602427 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878234aa-9cfd-4167-8c28-43a481c05cd5 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:26:12.717925 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 597/1177] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Aug 24 05:26:12 2022] POST /identity/v3/users => generated 311 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:26:14.682015 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d153a7b-25e3-4617-bf47-c4ed0d47199d None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:14.682579 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d153a7b-25e3-4617-bf47-c4ed0d47199d None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:14.682692 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d153a7b-25e3-4617-bf47-c4ed0d47199d None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:14.683937 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 581/1178] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:26:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:26:14.690010 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1920e-2fd0-4a85-9bf5-569d0d6d50cb None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:14.690404 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1920e-2fd0-4a85-9bf5-569d0d6d50cb None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:14.690563 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1920e-2fd0-4a85-9bf5-569d0d6d50cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:14.708619 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-fcb1920e-2fd0-4a85-9bf5-569d0d6d50cb None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:26:14.709806 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-fcb1920e-2fd0-4a85-9bf5-569d0d6d50cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:26:14.774786 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 598/1179] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:26:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:26:15.047692 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d153a7b-25e3-4617-bf47-c4ed0d47199d None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:26:15.133758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:26:15.149940 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d153a7b-25e3-4617-bf47-c4ed0d47199d None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:26:15.185989 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29271ab-de8f-47a8-a67c-0205c155e7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:26:15.186621 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29271ab-de8f-47a8-a67c-0205c155e7ea None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:15.186905 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29271ab-de8f-47a8-a67c-0205c155e7ea None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:15.187180 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29271ab-de8f-47a8-a67c-0205c155e7ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:15.197349 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29271ab-de8f-47a8-a67c-0205c155e7ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:26:15.203098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-c29271ab-de8f-47a8-a67c-0205c155e7ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:26:15.203098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:26:15.203098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:26:15.203098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:26:15.203098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:26:15.203098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:26:15.203098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:26:15.203098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:26:15.203098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:26:15.203098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:26:15.203098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:26:15.203098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:26:15.203098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:26:15.203098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:26:15.203098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:26:15.203098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:26:15.204310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:26:15.204310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:26:15.204310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:26:15.204310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:26:15.204310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:26:15.204310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:26:15.204310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:26:15.204310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:26:15.204310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:26:15.204310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:26:15.204310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:26:15.204310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:26:15.204310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:26:15.204310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:26:15.204310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:26:15.204310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:26:15.205257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:26:15.205257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:26:15.205257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:26:15.205257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:26:15.205257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:26:15.205257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:26:15.205257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:26:15.205257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:26:15.205257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:26:15.205257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:26:15.205257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:26:15.205257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:26:15.205257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 582/1180] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Aug 24 05:26:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:26:15.210373 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb1920e-2fd0-4a85-9bf5-569d0d6d50cb None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:26:15.281366 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:26:15.305544 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb1920e-2fd0-4a85-9bf5-569d0d6d50cb None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:26:15.338254 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9386a90-1211-428c-a45c-8c0034ce95d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:26:15.338876 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9386a90-1211-428c-a45c-8c0034ce95d2 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:15.340041 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9386a90-1211-428c-a45c-8c0034ce95d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:15.340041 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9386a90-1211-428c-a45c-8c0034ce95d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:15.340240 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9386a90-1211-428c-a45c-8c0034ce95d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:26:15.340472 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9386a90-1211-428c-a45c-8c0034ce95d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:26:15.341827 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9386a90-1211-428c-a45c-8c0034ce95d2 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:26:15.349468 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 599/1181] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:26:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:26:15.356424 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29271ab-de8f-47a8-a67c-0205c155e7ea None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:26:15.456246 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29271ab-de8f-47a8-a67c-0205c155e7ea None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:26:15.489733 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08227167-0b40-4ae6-bb40-7e284b028bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:26:15.490381 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08227167-0b40-4ae6-bb40-7e284b028bef None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:15.490661 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08227167-0b40-4ae6-bb40-7e284b028bef None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:15.490937 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08227167-0b40-4ae6-bb40-7e284b028bef None admin] PATH_INFO: `/v3/users/8d06272166d94121bcd1144171aea5ed` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:15.504233 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08227167-0b40-4ae6-bb40-7e284b028bef None admin] RBAC: Authorizing `identity:get_user(user_id=8d06272166d94121bcd1144171aea5ed)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:26:15.505695 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08227167-0b40-4ae6-bb40-7e284b028bef None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:26:15.507283 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 583/1182] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:26:15 2022] GET /identity/v3/users/8d06272166d94121bcd1144171aea5ed => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:26:15.513483 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9386a90-1211-428c-a45c-8c0034ce95d2 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:26:15.611290 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9386a90-1211-428c-a45c-8c0034ce95d2 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:26:15.644330 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e845d-2114-4c1d-81d4-2d94cbfa5c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:26:15.644973 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e845d-2114-4c1d-81d4-2d94cbfa5c42 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:15.645256 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e845d-2114-4c1d-81d4-2d94cbfa5c42 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:15.645533 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e845d-2114-4c1d-81d4-2d94cbfa5c42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:15.648919 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:26:15.710952 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53e845d-2114-4c1d-81d4-2d94cbfa5c42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:26:15.712390 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53e845d-2114-4c1d-81d4-2d94cbfa5c42 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:26:15.811753 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 600/1183] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:26:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:26:15.818776 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08227167-0b40-4ae6-bb40-7e284b028bef None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:26:15.822614 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:26:15.917856 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08227167-0b40-4ae6-bb40-7e284b028bef None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:26:15.950272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8660267-3614-41bb-a0c7-19f3dc487d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:26:15.950890 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8660267-3614-41bb-a0c7-19f3dc487d40 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:15.951169 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8660267-3614-41bb-a0c7-19f3dc487d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:15.951445 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8660267-3614-41bb-a0c7-19f3dc487d40 None admin] PATH_INFO: `/v3/projects/b084d9b5d0404536ac78b83dd7fabe5e` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:15.952588 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8660267-3614-41bb-a0c7-19f3dc487d40 None admin] RBAC: Authorizing `identity:get_project(project_id=b084d9b5d0404536ac78b83dd7fabe5e)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:26:15.954010 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8660267-3614-41bb-a0c7-19f3dc487d40 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:26:15.955620 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 584/1184] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:26:15 2022] GET /identity/v3/projects/b084d9b5d0404536ac78b83dd7fabe5e => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:26:15.961865 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e845d-2114-4c1d-81d4-2d94cbfa5c42 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:26:15.995501 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:26:16.058711 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e845d-2114-4c1d-81d4-2d94cbfa5c42 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:26:16.090864 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1622ac98-cfda-4186-badc-bef817d707dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:26:16.091450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622ac98-cfda-4186-badc-bef817d707dc None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:16.091728 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622ac98-cfda-4186-badc-bef817d707dc None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:16.091999 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622ac98-cfda-4186-badc-bef817d707dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:16.092824 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1622ac98-cfda-4186-badc-bef817d707dc None admin] RBAC: Adding query filter params (role.id=e013ad4d436f4cb0a9bf7debb7d13dc7, scope.project.id=b084d9b5d0404536ac78b83dd7fabe5e, user.id=8d06272166d94121bcd1144171aea5ed) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:26:16.093134 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1622ac98-cfda-4186-badc-bef817d707dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:26:16.094542 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1622ac98-cfda-4186-badc-bef817d707dc None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:26:16.099729 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 601/1185] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:26:15 2022] GET /identity/v3/role_assignments?user.id=8d06272166d94121bcd1144171aea5ed&scope.project.id=b084d9b5d0404536ac78b83dd7fabe5e&role.id=e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:26:18.042244 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a5053-436d-4b46-bd01-8b74387342b2 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:18.042813 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a5053-436d-4b46-bd01-8b74387342b2 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:18.042813 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a5053-436d-4b46-bd01-8b74387342b2 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:18.044136 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 585/1186] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:26:18.049306 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec83ad7-c91e-42dc-9e5f-5b38e263a43b None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:18.049727 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec83ad7-c91e-42dc-9e5f-5b38e263a43b None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:18.049864 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec83ad7-c91e-42dc-9e5f-5b38e263a43b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:18.068769 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-6ec83ad7-c91e-42dc-9e5f-5b38e263a43b None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:26:18.069988 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-6ec83ad7-c91e-42dc-9e5f-5b38e263a43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:26:18.135344 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 602/1187] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:26:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:26:18.404249 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a5053-436d-4b46-bd01-8b74387342b2 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:26:18.483003 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:26:18.515526 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a5053-436d-4b46-bd01-8b74387342b2 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:26:18.550438 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8119d5-fb0c-4202-afc7-c675326605d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:26:18.551059 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8119d5-fb0c-4202-afc7-c675326605d2 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:18.551341 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8119d5-fb0c-4202-afc7-c675326605d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:18.551616 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8119d5-fb0c-4202-afc7-c675326605d2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:18.562695 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8119d5-fb0c-4202-afc7-c675326605d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:26:18.567298 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-6f8119d5-fb0c-4202-afc7-c675326605d2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:26:18.567298 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:26:18.567298 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:26:18.567298 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:26:18.567298 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:26:18.567298 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:26:18.567298 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:26:18.567298 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:26:18.567298 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:26:18.567298 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:26:18.567298 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:26:18.567298 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:26:18.567298 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:26:18.567298 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:26:18.567298 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:26:18.567298 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:26:18.568447 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:26:18.568447 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:26:18.568447 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:26:18.568447 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:26:18.568447 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:26:18.568447 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:26:18.568447 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:26:18.568447 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:26:18.568447 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:26:18.568447 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:26:18.568447 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:26:18.568447 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:26:18.568447 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:26:18.568447 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:26:18.568447 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:26:18.568447 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:26:18.569570 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:26:18.569570 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:26:18.569570 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:26:18.569570 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:26:18.569570 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:26:18.569570 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:26:18.569570 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:26:18.569570 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:26:18.569570 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:26:18.569570 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:26:18.569570 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:26:18.569570 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:26:18.569570 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 586/1188] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Aug 24 05:26:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:26:18.575105 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec83ad7-c91e-42dc-9e5f-5b38e263a43b None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:26:18.604988 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:26:18.675778 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec83ad7-c91e-42dc-9e5f-5b38e263a43b None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:26:18.708814 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6521f9-b2ff-472f-8944-195e1b5def39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:26:18.709414 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6521f9-b2ff-472f-8944-195e1b5def39 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:18.709696 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6521f9-b2ff-472f-8944-195e1b5def39 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:18.709971 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6521f9-b2ff-472f-8944-195e1b5def39 None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:18.710632 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6521f9-b2ff-472f-8944-195e1b5def39 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:26:18.710939 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6521f9-b2ff-472f-8944-195e1b5def39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:26:18.712293 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6521f9-b2ff-472f-8944-195e1b5def39 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:26:18.721060 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 603/1189] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:26:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:26:18.727046 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8119d5-fb0c-4202-afc7-c675326605d2 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:26:18.837373 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8119d5-fb0c-4202-afc7-c675326605d2 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:26:18.868285 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:26:18.873207 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cddceb7-a4c5-4e59-85c6-411335cf70ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:26:18.873829 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cddceb7-a4c5-4e59-85c6-411335cf70ea None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:18.874113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cddceb7-a4c5-4e59-85c6-411335cf70ea None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:18.874399 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cddceb7-a4c5-4e59-85c6-411335cf70ea None admin] PATH_INFO: `/v3/users/8d06272166d94121bcd1144171aea5ed` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:18.875497 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cddceb7-a4c5-4e59-85c6-411335cf70ea None admin] RBAC: Authorizing `identity:get_user(user_id=8d06272166d94121bcd1144171aea5ed)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:26:18.876969 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cddceb7-a4c5-4e59-85c6-411335cf70ea None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:26:18.878540 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 587/1190] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:26:18 2022] GET /identity/v3/users/8d06272166d94121bcd1144171aea5ed => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:26:18.884617 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6521f9-b2ff-472f-8944-195e1b5def39 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:26:18.976000 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:26:18.984898 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6521f9-b2ff-472f-8944-195e1b5def39 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:26:19.018050 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebc88b-1b41-41e1-b5d4-00f7a7392827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:26:19.018691 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebc88b-1b41-41e1-b5d4-00f7a7392827 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:19.018988 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebc88b-1b41-41e1-b5d4-00f7a7392827 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:19.019267 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebc88b-1b41-41e1-b5d4-00f7a7392827 None admin] PATH_INFO: `/v3/projects/b084d9b5d0404536ac78b83dd7fabe5e` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:19.020433 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ebc88b-1b41-41e1-b5d4-00f7a7392827 None admin] RBAC: Authorizing `identity:get_project(project_id=b084d9b5d0404536ac78b83dd7fabe5e)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:26:19.021868 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ebc88b-1b41-41e1-b5d4-00f7a7392827 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:26:19.023539 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 604/1191] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:26:18 2022] GET /identity/v3/projects/b084d9b5d0404536ac78b83dd7fabe5e => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:26:19.029614 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cddceb7-a4c5-4e59-85c6-411335cf70ea None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:26:19.139017 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cddceb7-a4c5-4e59-85c6-411335cf70ea None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:26:19.173493 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247175d8-4b95-49a1-b16e-f6b467959313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:26:19.174121 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247175d8-4b95-49a1-b16e-f6b467959313 None admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:19.174404 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247175d8-4b95-49a1-b16e-f6b467959313 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:19.174681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247175d8-4b95-49a1-b16e-f6b467959313 None admin] PATH_INFO: `/v3/projects/b084d9b5d0404536ac78b83dd7fabe5e/users/8d06272166d94121bcd1144171aea5ed/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:19.176647 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247175d8-4b95-49a1-b16e-f6b467959313 None admin] RBAC: Authorizing `identity:create_grant(project_id=b084d9b5d0404536ac78b83dd7fabe5e, user_id=8d06272166d94121bcd1144171aea5ed, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:26:19.178072 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247175d8-4b95-49a1-b16e-f6b467959313 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:26:19.249629 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 588/1192] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Aug 24 05:26:19 2022] PUT /identity/v3/projects/b084d9b5d0404536ac78b83dd7fabe5e/users/8d06272166d94121bcd1144171aea5ed/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:26:21.219549 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60684ea0-27a4-482d-a7a6-89813c1dab7e None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:21.220021 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60684ea0-27a4-482d-a7a6-89813c1dab7e None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:21.220129 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60684ea0-27a4-482d-a7a6-89813c1dab7e None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:21.221462 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 605/1193] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:26:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:26:21.226926 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4cc4c8-1ead-4113-b34c-8174bbf9ae90 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:21.227323 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4cc4c8-1ead-4113-b34c-8174bbf9ae90 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:21.227485 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4cc4c8-1ead-4113-b34c-8174bbf9ae90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:21.245882 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-7b4cc4c8-1ead-4113-b34c-8174bbf9ae90 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:26:21.247076 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-7b4cc4c8-1ead-4113-b34c-8174bbf9ae90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:26:21.303768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:26:21.315694 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 589/1194] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:26:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:26:21.584606 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60684ea0-27a4-482d-a7a6-89813c1dab7e None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:26:21.688119 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60684ea0-27a4-482d-a7a6-89813c1dab7e None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:26:21.722241 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9393026-5402-4013-b8b3-0e6453393bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:26:21.722866 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9393026-5402-4013-b8b3-0e6453393bc2 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:21.723148 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9393026-5402-4013-b8b3-0e6453393bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:21.723424 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9393026-5402-4013-b8b3-0e6453393bc2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:21.733533 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9393026-5402-4013-b8b3-0e6453393bc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:26:21.738415 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-e9393026-5402-4013-b8b3-0e6453393bc2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:26:21.738415 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:26:21.738415 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:26:21.738415 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:26:21.738415 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:26:21.738415 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:26:21.738415 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:26:21.738415 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:26:21.738415 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:26:21.738415 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:26:21.738415 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:26:21.738415 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:26:21.738415 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:26:21.738415 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:26:21.738415 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:26:21.738415 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:26:21.739492 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:26:21.739492 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:26:21.739492 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:26:21.739492 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:26:21.739492 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:26:21.739492 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:26:21.739492 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:26:21.739492 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:26:21.739492 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:26:21.739492 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:26:21.739492 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:26:21.739492 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:26:21.739492 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:26:21.739492 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:26:21.739492 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:26:21.739492 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:26:21.740764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:26:21.740764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:26:21.740764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:26:21.740764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:26:21.740764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:26:21.740764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:26:21.740764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:26:21.740764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:26:21.740764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:26:21.740764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:26:21.740764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:26:21.740764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:26:21.740764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 606/1195] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Aug 24 05:26:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:26:21.746412 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4cc4c8-1ead-4113-b34c-8174bbf9ae90 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:26:21.848592 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4cc4c8-1ead-4113-b34c-8174bbf9ae90 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:26:21.887150 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca08ef4-3cbb-4803-9913-4691dff7dae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:26:21.887754 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca08ef4-3cbb-4803-9913-4691dff7dae0 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:21.888041 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca08ef4-3cbb-4803-9913-4691dff7dae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:21.888318 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca08ef4-3cbb-4803-9913-4691dff7dae0 None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:21.889008 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca08ef4-3cbb-4803-9913-4691dff7dae0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:26:21.889317 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca08ef4-3cbb-4803-9913-4691dff7dae0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:26:21.890681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca08ef4-3cbb-4803-9913-4691dff7dae0 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:26:21.898538 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 590/1196] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:26:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:26:21.905020 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9393026-5402-4013-b8b3-0e6453393bc2 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:26:21.928410 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:26:22.002723 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9393026-5402-4013-b8b3-0e6453393bc2 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:26:22.035629 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ec901-0aea-4b27-b110-3fbe4dca2da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:26:22.036246 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ec901-0aea-4b27-b110-3fbe4dca2da2 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:22.036555 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ec901-0aea-4b27-b110-3fbe4dca2da2 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:22.036830 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ec901-0aea-4b27-b110-3fbe4dca2da2 None admin] PATH_INFO: `/v3/users/8d06272166d94121bcd1144171aea5ed` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:22.037947 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ec901-0aea-4b27-b110-3fbe4dca2da2 None admin] RBAC: Authorizing `identity:get_user(user_id=8d06272166d94121bcd1144171aea5ed)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:26:22.039375 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ec901-0aea-4b27-b110-3fbe4dca2da2 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:26:22.040967 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 607/1197] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Aug 24 05:26:21 2022] GET /identity/v3/users/8d06272166d94121bcd1144171aea5ed => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:26:22.047807 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca08ef4-3cbb-4803-9913-4691dff7dae0 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:26:22.099513 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:26:22.149049 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca08ef4-3cbb-4803-9913-4691dff7dae0 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:26:22.183615 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfff962a-aa56-4149-94bc-25fbd15612da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:26:22.184219 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff962a-aa56-4149-94bc-25fbd15612da None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:22.184532 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff962a-aa56-4149-94bc-25fbd15612da None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:22.184807 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff962a-aa56-4149-94bc-25fbd15612da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:22.250492 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfff962a-aa56-4149-94bc-25fbd15612da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:26:22.251930 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfff962a-aa56-4149-94bc-25fbd15612da None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:26:22.301119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:26:22.350506 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 591/1198] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:26:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:26:22.356669 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ec901-0aea-4b27-b110-3fbe4dca2da2 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:26:22.442544 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:26:22.454797 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ec901-0aea-4b27-b110-3fbe4dca2da2 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:26:22.487638 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45965694-a120-49e9-bdbf-0c6da6dab5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:26:22.488262 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45965694-a120-49e9-bdbf-0c6da6dab5ae None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:22.488580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45965694-a120-49e9-bdbf-0c6da6dab5ae None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:22.488857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45965694-a120-49e9-bdbf-0c6da6dab5ae None admin] PATH_INFO: `/v3/projects/b084d9b5d0404536ac78b83dd7fabe5e` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:22.489967 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45965694-a120-49e9-bdbf-0c6da6dab5ae None admin] RBAC: Authorizing `identity:get_project(project_id=b084d9b5d0404536ac78b83dd7fabe5e)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:26:22.491388 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45965694-a120-49e9-bdbf-0c6da6dab5ae None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:26:22.493031 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 608/1199] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Aug 24 05:26:22 2022] GET /identity/v3/projects/b084d9b5d0404536ac78b83dd7fabe5e => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:26:22.499860 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfff962a-aa56-4149-94bc-25fbd15612da None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:26:22.598567 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfff962a-aa56-4149-94bc-25fbd15612da None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:26:22.632664 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcedd1b-6037-4b64-a87d-d840f9a78330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:26:22.633266 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcedd1b-6037-4b64-a87d-d840f9a78330 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:26:22.633548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcedd1b-6037-4b64-a87d-d840f9a78330 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:26:22.633824 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcedd1b-6037-4b64-a87d-d840f9a78330 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:26:22.634556 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcedd1b-6037-4b64-a87d-d840f9a78330 None admin] RBAC: Adding query filter params (role.id=e013ad4d436f4cb0a9bf7debb7d13dc7, scope.project.id=b084d9b5d0404536ac78b83dd7fabe5e, user.id=8d06272166d94121bcd1144171aea5ed) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:26:22.634869 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcedd1b-6037-4b64-a87d-d840f9a78330 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:26:22.637114 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcedd1b-6037-4b64-a87d-d840f9a78330 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:26:22.642868 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 592/1200] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:26:22 2022] GET /identity/v3/role_assignments?user.id=8d06272166d94121bcd1144171aea5ed&scope.project.id=b084d9b5d0404536ac78b83dd7fabe5e&role.id=e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:26:32.483581 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=22 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:26:32.640243 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=92 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:27:43.965712 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c783b-8257-4d06-9739-81a1cddb71bb None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:43.966327 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c783b-8257-4d06-9739-81a1cddb71bb None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:43.966327 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c783b-8257-4d06-9739-81a1cddb71bb None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:43.967847 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 609/1201] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:27:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:27:43.974620 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b27005-5151-47bb-99a7-c8e8566187ba None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:43.976522 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b27005-5151-47bb-99a7-c8e8566187ba None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:43.976522 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b27005-5151-47bb-99a7-c8e8566187ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:43.996141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-40b27005-5151-47bb-99a7-c8e8566187ba None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:27:43.997460 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-40b27005-5151-47bb-99a7-c8e8566187ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:27:44.079383 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 593/1202] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:27:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:27:44.369886 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c783b-8257-4d06-9739-81a1cddb71bb None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:27:44.493048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c783b-8257-4d06-9739-81a1cddb71bb None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:27:44.526822 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc82685b-2637-4901-9821-dc47daa53fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:27:44.527449 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82685b-2637-4901-9821-dc47daa53fe8 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:44.527735 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82685b-2637-4901-9821-dc47daa53fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:44.528012 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82685b-2637-4901-9821-dc47daa53fe8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:44.539335 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc82685b-2637-4901-9821-dc47daa53fe8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:27:44.541003 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc82685b-2637-4901-9821-dc47daa53fe8 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:27:44.556109 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-cc82685b-2637-4901-9821-dc47daa53fe8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:27:44.556109 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:27:44.556109 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:27:44.556109 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:27:44.556109 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:27:44.556109 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:27:44.556109 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:27:44.556109 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:27:44.556109 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:27:44.556109 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:27:44.556109 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:27:44.556109 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:27:44.556109 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:27:44.556109 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:27:44.556109 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:27:44.556109 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:27:44.557512 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:27:44.557512 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:27:44.557512 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:27:44.557512 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:27:44.557512 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:27:44.557512 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:27:44.557512 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:27:44.557512 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:27:44.557512 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:27:44.557512 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:27:44.557512 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:27:44.557512 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:27:44.557512 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:27:44.557512 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:27:44.557512 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:27:44.557512 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:27:44.557512 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:27:44.558845 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:27:44.558845 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:27:44.558845 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:27:44.558845 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 610/1203] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:27:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:27:44.563415 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b27005-5151-47bb-99a7-c8e8566187ba None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:27:44.670130 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b27005-5151-47bb-99a7-c8e8566187ba None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:27:44.699710 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:27:44.705474 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8246bef-9f2e-4cdf-bdec-de01fef18e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:27:44.706749 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8246bef-9f2e-4cdf-bdec-de01fef18e76 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:44.707176 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8246bef-9f2e-4cdf-bdec-de01fef18e76 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:44.707455 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8246bef-9f2e-4cdf-bdec-de01fef18e76 None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:44.708104 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8246bef-9f2e-4cdf-bdec-de01fef18e76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:27:44.708425 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8246bef-9f2e-4cdf-bdec-de01fef18e76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:27:44.709820 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8246bef-9f2e-4cdf-bdec-de01fef18e76 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:27:44.720886 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 594/1204] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:27:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:27:44.727536 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc82685b-2637-4901-9821-dc47daa53fe8 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:27:44.785494 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:27:44.833821 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc82685b-2637-4901-9821-dc47daa53fe8 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:27:44.868805 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df8f02c-fe21-4126-9c39-6de5b28d6697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:27:44.869392 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8f02c-fe21-4126-9c39-6de5b28d6697 None admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:44.869676 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8f02c-fe21-4126-9c39-6de5b28d6697 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:44.869961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8f02c-fe21-4126-9c39-6de5b28d6697 None admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:44.870600 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df8f02c-fe21-4126-9c39-6de5b28d6697 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:27:44.872135 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df8f02c-fe21-4126-9c39-6de5b28d6697 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:27:45.119858 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 611/1205] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Aug 24 05:27:44 2022] POST /identity/v3/users => generated 251 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:27:47.097349 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c22753f-3538-4473-b64b-5b697394ebdf None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:47.098181 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c22753f-3538-4473-b64b-5b697394ebdf None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:47.098350 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c22753f-3538-4473-b64b-5b697394ebdf None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:47.099711 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 595/1206] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:27:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:27:47.105012 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c25728-636f-41b0-976e-987378349f48 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:47.105463 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c25728-636f-41b0-976e-987378349f48 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:47.105573 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c25728-636f-41b0-976e-987378349f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:47.124180 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-b6c25728-636f-41b0-976e-987378349f48 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:27:47.125394 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-b6c25728-636f-41b0-976e-987378349f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:27:47.193861 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 612/1207] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:27:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:27:47.467670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c22753f-3538-4473-b64b-5b697394ebdf None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:27:47.569116 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c22753f-3538-4473-b64b-5b697394ebdf None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:27:47.602590 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c46f5-1033-49d4-880f-1765175de1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:27:47.603241 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c46f5-1033-49d4-880f-1765175de1e9 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:47.603521 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c46f5-1033-49d4-880f-1765175de1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:47.603793 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c46f5-1033-49d4-880f-1765175de1e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:47.614221 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420c46f5-1033-49d4-880f-1765175de1e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:27:47.619002 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-420c46f5-1033-49d4-880f-1765175de1e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:27:47.619002 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:27:47.619002 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:27:47.619002 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:27:47.619002 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:27:47.619002 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:27:47.619002 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:27:47.619002 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:27:47.619002 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:27:47.619002 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:27:47.619002 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:27:47.619002 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:27:47.619002 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:27:47.619002 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:27:47.619002 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:27:47.619002 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:27:47.624141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:27:47.624141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:27:47.624141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:27:47.624141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:27:47.624141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:27:47.624141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:27:47.624141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:27:47.624141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:27:47.624141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:27:47.624141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:27:47.624141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:27:47.624141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:27:47.624141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:27:47.624141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:27:47.624141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:27:47.624141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:27:47.625271 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:27:47.625271 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:27:47.625271 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:27:47.625271 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:27:47.625271 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:27:47.625271 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:27:47.625271 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:27:47.625271 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:27:47.625271 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:27:47.625271 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:27:47.625271 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:27:47.625271 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:27:47.625271 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 596/1208] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Aug 24 05:27:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:27:47.628741 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c25728-636f-41b0-976e-987378349f48 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:27:47.667477 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:27:47.727143 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c25728-636f-41b0-976e-987378349f48 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:27:47.761714 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca1d70-a047-4e83-b370-a813fcbf1b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:27:47.762319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca1d70-a047-4e83-b370-a813fcbf1b38 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:47.762601 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca1d70-a047-4e83-b370-a813fcbf1b38 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:47.762877 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca1d70-a047-4e83-b370-a813fcbf1b38 None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:47.763542 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca1d70-a047-4e83-b370-a813fcbf1b38 None admin] RBAC: Adding query filter params (name=service) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:27:47.763848 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca1d70-a047-4e83-b370-a813fcbf1b38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:27:47.765241 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca1d70-a047-4e83-b370-a813fcbf1b38 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:27:47.773588 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 613/1209] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Aug 24 05:27:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:27:47.779388 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c46f5-1033-49d4-880f-1765175de1e9 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:27:47.829510 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:27:47.878273 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c46f5-1033-49d4-880f-1765175de1e9 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:27:47.910729 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb519f7-f503-4fcc-a46b-b593d1aaf4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:27:47.911350 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb519f7-f503-4fcc-a46b-b593d1aaf4e7 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:47.911632 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb519f7-f503-4fcc-a46b-b593d1aaf4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:47.911906 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb519f7-f503-4fcc-a46b-b593d1aaf4e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:47.922063 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb519f7-f503-4fcc-a46b-b593d1aaf4e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:27:47.923530 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb519f7-f503-4fcc-a46b-b593d1aaf4e7 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:27:47.938254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-4bb519f7-f503-4fcc-a46b-b593d1aaf4e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:27:47.938254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:27:47.938254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:27:47.938254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:27:47.938254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:27:47.938254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:27:47.938254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:27:47.938254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:27:47.938254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:27:47.938254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:27:47.938254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:27:47.938254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:27:47.938254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:27:47.938254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:27:47.938254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:27:47.938254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:27:47.939446 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:27:47.939446 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:27:47.939446 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:27:47.939446 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:27:47.939446 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:27:47.939446 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:27:47.939446 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:27:47.939446 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:27:47.939446 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:27:47.939446 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:27:47.939446 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:27:47.939446 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:27:47.939446 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:27:47.939446 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:27:47.939446 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:27:47.939446 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:27:47.939446 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:27:47.941306 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:27:47.941306 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:27:47.941306 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:27:47.941306 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 597/1210] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:27:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:27:47.946310 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca1d70-a047-4e83-b370-a813fcbf1b38 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:27:48.042276 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:27:48.045058 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca1d70-a047-4e83-b370-a813fcbf1b38 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:27:48.078841 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25be52e2-87ed-4ce6-9ae6-5db309ce3176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:27:48.079453 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be52e2-87ed-4ce6-9ae6-5db309ce3176 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:48.079737 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be52e2-87ed-4ce6-9ae6-5db309ce3176 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:48.080013 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be52e2-87ed-4ce6-9ae6-5db309ce3176 None admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:48.080770 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25be52e2-87ed-4ce6-9ae6-5db309ce3176 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:27:48.081149 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25be52e2-87ed-4ce6-9ae6-5db309ce3176 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:27:48.082561 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25be52e2-87ed-4ce6-9ae6-5db309ce3176 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:27:48.094096 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 614/1211] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:27:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:27:48.108817 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb519f7-f503-4fcc-a46b-b593d1aaf4e7 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:27:48.212317 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb519f7-f503-4fcc-a46b-b593d1aaf4e7 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:27:48.213858 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:27:48.246341 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23a7f5c-a366-4b4b-8f9b-fdddd926c80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:27:48.246961 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a7f5c-a366-4b4b-8f9b-fdddd926c80f None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:48.247245 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a7f5c-a366-4b4b-8f9b-fdddd926c80f None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:48.247522 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a7f5c-a366-4b4b-8f9b-fdddd926c80f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:48.255072 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23a7f5c-a366-4b4b-8f9b-fdddd926c80f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:27:48.256530 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23a7f5c-a366-4b4b-8f9b-fdddd926c80f None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:27:48.267594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-c23a7f5c-a366-4b4b-8f9b-fdddd926c80f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 05:27:48.267594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:27:48.267594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:27:48.267594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:27:48.267594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:27:48.267594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:27:48.267594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:27:48.267594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:27:48.267594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:27:48.267594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:27:48.267594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:27:48.267594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:27:48.267594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:27:48.267594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:27:48.267594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:27:48.267594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:27:48.268737 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:27:48.268737 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:27:48.268737 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:27:48.268737 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:27:48.268737 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:27:48.268737 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:27:48.268737 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:27:48.268737 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:27:48.268737 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:27:48.268737 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:27:48.268737 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:27:48.268737 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:27:48.268737 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:27:48.268737 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:27:48.268737 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:27:48.268737 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:27:48.268737 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:27:48.268737 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:27:48.269934 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:27:48.269934 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:27:48.269934 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:27:48.269934 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:27:48.269934 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:27:48.269934 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:27:48.269934 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:27:48.269934 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:27:48.269934 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 05:27:48.269934 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:27:48.269934 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 598/1212] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:27:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:27:48.274955 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25be52e2-87ed-4ce6-9ae6-5db309ce3176 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:27:48.375227 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25be52e2-87ed-4ce6-9ae6-5db309ce3176 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:27:48.413712 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1524af26-9d49-4060-b6e8-b06378496908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:27:48.414330 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524af26-9d49-4060-b6e8-b06378496908 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:48.414611 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524af26-9d49-4060-b6e8-b06378496908 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:48.414886 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524af26-9d49-4060-b6e8-b06378496908 None admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:48.415622 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1524af26-9d49-4060-b6e8-b06378496908 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:27:48.415925 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1524af26-9d49-4060-b6e8-b06378496908 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:27:48.418240 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1524af26-9d49-4060-b6e8-b06378496908 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:27:48.436703 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 615/1213] 10.222.0.15 () {56 vars in 1143 bytes} [Wed Aug 24 05:27:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:27:48.442613 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23a7f5c-a366-4b4b-8f9b-fdddd926c80f None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:27:48.560526 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23a7f5c-a366-4b4b-8f9b-fdddd926c80f None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:27:48.595021 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e724d-e635-46bd-82f9-6a314d5e724d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:27:48.595715 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e724d-e635-46bd-82f9-6a314d5e724d None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:48.595993 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e724d-e635-46bd-82f9-6a314d5e724d None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:48.596880 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e724d-e635-46bd-82f9-6a314d5e724d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:48.628160 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:27:48.669945 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1e724d-e635-46bd-82f9-6a314d5e724d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:27:48.669945 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1e724d-e635-46bd-82f9-6a314d5e724d None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:27:48.763745 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 599/1214] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:27:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:27:48.770845 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1524af26-9d49-4060-b6e8-b06378496908 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:27:48.791207 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:27:48.871636 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1524af26-9d49-4060-b6e8-b06378496908 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:27:48.909422 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15be4b0-1c40-4c20-9094-2aeeb46b14b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:27:48.909422 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15be4b0-1c40-4c20-9094-2aeeb46b14b4 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:48.909422 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15be4b0-1c40-4c20-9094-2aeeb46b14b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:48.909422 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15be4b0-1c40-4c20-9094-2aeeb46b14b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:48.916160 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15be4b0-1c40-4c20-9094-2aeeb46b14b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:27:48.917624 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15be4b0-1c40-4c20-9094-2aeeb46b14b4 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:27:48.931727 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-c15be4b0-1c40-4c20-9094-2aeeb46b14b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:27:48.931727 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:27:48.931727 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:27:48.931727 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:27:48.931727 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:27:48.931727 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:27:48.931727 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:27:48.931727 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:27:48.931727 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:27:48.931727 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:27:48.931727 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:27:48.931727 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:27:48.931727 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:27:48.931727 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:27:48.931727 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:27:48.931727 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:27:48.932787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:27:48.932787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:27:48.932787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:27:48.932787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:27:48.932787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:27:48.932787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:27:48.932787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:27:48.932787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:27:48.932787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:27:48.932787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:27:48.932787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:27:48.932787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:27:48.932787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:27:48.932787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:27:48.932787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:27:48.932787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:27:48.932787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:27:48.933744 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:27:48.933744 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:27:48.933744 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:27:48.933744 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 616/1215] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:27:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:27:48.940234 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e724d-e635-46bd-82f9-6a314d5e724d None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:27:49.000168 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:27:49.040137 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e724d-e635-46bd-82f9-6a314d5e724d None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:27:49.072281 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca556c3-d522-42cf-a811-4d904a58f0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:27:49.072903 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca556c3-d522-42cf-a811-4d904a58f0ec None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:49.073187 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca556c3-d522-42cf-a811-4d904a58f0ec None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:49.073460 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca556c3-d522-42cf-a811-4d904a58f0ec None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:49.074091 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca556c3-d522-42cf-a811-4d904a58f0ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:27:49.074399 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca556c3-d522-42cf-a811-4d904a58f0ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:27:49.075796 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca556c3-d522-42cf-a811-4d904a58f0ec None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:27:49.087690 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 600/1216] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:27:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:27:49.093726 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15be4b0-1c40-4c20-9094-2aeeb46b14b4 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:27:49.167986 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:27:49.199055 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15be4b0-1c40-4c20-9094-2aeeb46b14b4 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:27:49.232264 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf2a72-e584-49f4-9171-24ab644deb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:27:49.232974 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf2a72-e584-49f4-9171-24ab644deb17 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:49.233267 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf2a72-e584-49f4-9171-24ab644deb17 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:49.233533 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf2a72-e584-49f4-9171-24ab644deb17 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:49.239973 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcf2a72-e584-49f4-9171-24ab644deb17 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:27:49.241446 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcf2a72-e584-49f4-9171-24ab644deb17 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:27:49.252141 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-4bcf2a72-e584-49f4-9171-24ab644deb17 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:27:49.252141 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:27:49.252141 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:27:49.252141 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:27:49.252141 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:27:49.252141 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:27:49.252141 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:27:49.252141 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:27:49.252141 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:27:49.252141 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:27:49.252141 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:27:49.252141 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:27:49.252141 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:27:49.252141 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:27:49.252141 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:27:49.256292 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:27:49.256292 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:27:49.256292 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:27:49.256292 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:27:49.256292 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:27:49.256292 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:27:49.256292 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:27:49.256292 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:27:49.256292 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:27:49.256292 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:27:49.256292 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:27:49.256292 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:27:49.256292 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:27:49.256292 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:27:49.256292 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:27:49.256292 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:27:49.256292 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:27:49.257428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:27:49.257428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:27:49.257428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:27:49.257428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:27:49.257428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:27:49.257428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:27:49.257428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:27:49.257428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:27:49.257428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 617/1217] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:27:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:27:49.259932 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca556c3-d522-42cf-a811-4d904a58f0ec None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:27:49.362024 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca556c3-d522-42cf-a811-4d904a58f0ec None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:27:49.387616 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:27:49.399909 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaf5a9c-00f3-4016-82be-656f41da5eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:27:49.400553 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf5a9c-00f3-4016-82be-656f41da5eae None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:49.400876 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf5a9c-00f3-4016-82be-656f41da5eae None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:49.401229 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf5a9c-00f3-4016-82be-656f41da5eae None admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:49.401932 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaf5a9c-00f3-4016-82be-656f41da5eae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:27:49.402238 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaf5a9c-00f3-4016-82be-656f41da5eae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:27:49.403662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaf5a9c-00f3-4016-82be-656f41da5eae None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:27:49.415663 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 601/1218] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Aug 24 05:27:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:27:49.421943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf2a72-e584-49f4-9171-24ab644deb17 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:27:49.526066 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf2a72-e584-49f4-9171-24ab644deb17 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:27:49.555136 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:27:49.560850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d0ca5-be21-448f-8221-9f3453766c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:27:49.561444 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d0ca5-be21-448f-8221-9f3453766c3e None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:49.561803 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d0ca5-be21-448f-8221-9f3453766c3e None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:49.562083 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d0ca5-be21-448f-8221-9f3453766c3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:49.562797 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26d0ca5-be21-448f-8221-9f3453766c3e None admin] RBAC: Adding query filter params (role.id=cb40cd8499f7418ca0da5ea0f1b87ff9, scope.project.id=59592706ae9442e08353158c625e9581, user.id=6778b7c5ad19484b877494f143050405) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:27:49.563102 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26d0ca5-be21-448f-8221-9f3453766c3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:27:49.564540 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26d0ca5-be21-448f-8221-9f3453766c3e None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:27:49.570238 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 618/1219] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:27:49 2022] GET /identity/v3/role_assignments?user.id=6778b7c5ad19484b877494f143050405&scope.project.id=59592706ae9442e08353158c625e9581&role.id=cb40cd8499f7418ca0da5ea0f1b87ff9 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:27:51.527473 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebfeb86-6e6d-41df-afa7-c242b2b1e1b1 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:51.527989 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebfeb86-6e6d-41df-afa7-c242b2b1e1b1 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:51.527989 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebfeb86-6e6d-41df-afa7-c242b2b1e1b1 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:51.529356 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 602/1220] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:27:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:27:51.535720 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed52eb6-83f6-450f-b515-24f18074c77c None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:51.536127 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed52eb6-83f6-450f-b515-24f18074c77c None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:51.536306 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed52eb6-83f6-450f-b515-24f18074c77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:51.555223 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-fed52eb6-83f6-450f-b515-24f18074c77c None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:27:51.556441 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-fed52eb6-83f6-450f-b515-24f18074c77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:27:51.625260 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 619/1221] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:27:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:27:51.913281 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebfeb86-6e6d-41df-afa7-c242b2b1e1b1 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:27:52.014126 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebfeb86-6e6d-41df-afa7-c242b2b1e1b1 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:27:52.049286 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd62bf-5bb2-4d73-8419-41df046dd679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:27:52.049929 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd62bf-5bb2-4d73-8419-41df046dd679 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:52.050210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd62bf-5bb2-4d73-8419-41df046dd679 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:52.050486 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd62bf-5bb2-4d73-8419-41df046dd679 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:52.060938 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacd62bf-5bb2-4d73-8419-41df046dd679 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:27:52.065734 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-aacd62bf-5bb2-4d73-8419-41df046dd679 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:27:52.065734 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:27:52.065734 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:27:52.065734 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:27:52.065734 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:27:52.065734 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:27:52.065734 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:27:52.065734 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:27:52.065734 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:27:52.065734 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:27:52.065734 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:27:52.065734 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:27:52.065734 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:27:52.065734 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:27:52.065734 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:27:52.065734 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:27:52.066820 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:27:52.066820 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:27:52.066820 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:27:52.066820 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:27:52.066820 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:27:52.066820 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:27:52.066820 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:27:52.066820 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:27:52.066820 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:27:52.066820 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:27:52.066820 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:27:52.066820 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:27:52.066820 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:27:52.066820 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:27:52.066820 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:27:52.066820 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:27:52.067713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:27:52.067713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:27:52.067713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:27:52.067713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:27:52.067713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:27:52.067713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:27:52.067713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:27:52.067713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:27:52.067713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:27:52.067713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:27:52.067713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:27:52.067713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:27:52.067713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 603/1222] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Aug 24 05:27:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:27:52.072956 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed52eb6-83f6-450f-b515-24f18074c77c None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:27:52.173421 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed52eb6-83f6-450f-b515-24f18074c77c None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:27:52.199108 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:27:52.209859 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b2a7d3-612d-4536-9ac8-4a30c8440cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:27:52.210468 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b2a7d3-612d-4536-9ac8-4a30c8440cb9 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:52.210751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b2a7d3-612d-4536-9ac8-4a30c8440cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:52.211029 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b2a7d3-612d-4536-9ac8-4a30c8440cb9 None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:52.211696 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b2a7d3-612d-4536-9ac8-4a30c8440cb9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:27:52.212007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b2a7d3-612d-4536-9ac8-4a30c8440cb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:27:52.213409 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b2a7d3-612d-4536-9ac8-4a30c8440cb9 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:27:52.221741 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 620/1223] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Aug 24 05:27:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:27:52.227575 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd62bf-5bb2-4d73-8419-41df046dd679 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:27:52.271986 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:27:52.327101 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd62bf-5bb2-4d73-8419-41df046dd679 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:27:52.361075 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d004ed-c1d0-462d-badb-0b19fbc1eef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:27:52.362298 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d004ed-c1d0-462d-badb-0b19fbc1eef3 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:52.362567 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d004ed-c1d0-462d-badb-0b19fbc1eef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:52.362845 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d004ed-c1d0-462d-badb-0b19fbc1eef3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:52.372838 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d004ed-c1d0-462d-badb-0b19fbc1eef3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:27:52.374345 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d004ed-c1d0-462d-badb-0b19fbc1eef3 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:27:52.388781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-12d004ed-c1d0-462d-badb-0b19fbc1eef3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:27:52.388781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:27:52.388781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:27:52.388781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:27:52.388781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:27:52.388781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:27:52.388781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:27:52.388781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:27:52.388781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:27:52.388781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:27:52.388781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:27:52.388781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:27:52.388781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:27:52.388781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:27:52.388781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:27:52.388781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:27:52.389981 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:27:52.389981 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:27:52.389981 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:27:52.389981 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:27:52.389981 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:27:52.389981 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:27:52.389981 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:27:52.389981 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:27:52.389981 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:27:52.389981 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:27:52.389981 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:27:52.389981 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:27:52.389981 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:27:52.389981 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:27:52.389981 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:27:52.389981 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:27:52.389981 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:27:52.391007 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:27:52.391007 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:27:52.391007 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:27:52.391007 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 604/1224] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:27:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:27:52.395826 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b2a7d3-612d-4536-9ac8-4a30c8440cb9 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:27:52.496670 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b2a7d3-612d-4536-9ac8-4a30c8440cb9 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:27:52.536357 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa9fd6-722a-47b2-b308-e65c8e4a7b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:27:52.537120 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa9fd6-722a-47b2-b308-e65c8e4a7b98 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:52.537270 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa9fd6-722a-47b2-b308-e65c8e4a7b98 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:52.537880 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa9fd6-722a-47b2-b308-e65c8e4a7b98 None admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:52.538525 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfa9fd6-722a-47b2-b308-e65c8e4a7b98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:27:52.538834 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfa9fd6-722a-47b2-b308-e65c8e4a7b98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:27:52.540296 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfa9fd6-722a-47b2-b308-e65c8e4a7b98 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:27:52.552028 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 621/1225] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:27:52.558104 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d004ed-c1d0-462d-badb-0b19fbc1eef3 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:27:52.656336 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:27:52.663882 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d004ed-c1d0-462d-badb-0b19fbc1eef3 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:27:52.697260 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e0740-53d2-41c1-a701-19bc76416399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:27:52.697946 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e0740-53d2-41c1-a701-19bc76416399 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:52.698226 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e0740-53d2-41c1-a701-19bc76416399 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:52.698501 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e0740-53d2-41c1-a701-19bc76416399 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:52.705315 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674e0740-53d2-41c1-a701-19bc76416399 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:27:52.707409 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674e0740-53d2-41c1-a701-19bc76416399 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:27:52.718573 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-674e0740-53d2-41c1-a701-19bc76416399 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 05:27:52.718573 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:27:52.718573 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:27:52.718573 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:27:52.718573 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:27:52.718573 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:27:52.718573 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:27:52.718573 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:27:52.718573 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:27:52.718573 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:27:52.718573 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:27:52.718573 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:27:52.718573 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:27:52.718573 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:27:52.718573 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:27:52.718573 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:27:52.719633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:27:52.719633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:27:52.719633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:27:52.719633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:27:52.719633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:27:52.719633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:27:52.719633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:27:52.719633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:27:52.719633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:27:52.719633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:27:52.719633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:27:52.719633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:27:52.719633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:27:52.719633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:27:52.719633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:27:52.719633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:27:52.719633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:27:52.719633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:27:52.720818 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:27:52.720818 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:27:52.720818 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:27:52.720818 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:27:52.720818 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:27:52.720818 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:27:52.720818 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:27:52.720818 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:27:52.720818 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 05:27:52.720818 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:27:52.720818 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 605/1226] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:27:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:27:52.726368 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa9fd6-722a-47b2-b308-e65c8e4a7b98 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:27:52.772961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:27:52.827600 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa9fd6-722a-47b2-b308-e65c8e4a7b98 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:27:52.866758 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a1c7cf-02a1-4aa1-be8c-0b71a1dcd078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:27:52.867357 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1c7cf-02a1-4aa1-be8c-0b71a1dcd078 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:52.867632 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1c7cf-02a1-4aa1-be8c-0b71a1dcd078 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:52.867899 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1c7cf-02a1-4aa1-be8c-0b71a1dcd078 None admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:52.868647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a1c7cf-02a1-4aa1-be8c-0b71a1dcd078 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:27:52.868950 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a1c7cf-02a1-4aa1-be8c-0b71a1dcd078 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:27:52.870360 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a1c7cf-02a1-4aa1-be8c-0b71a1dcd078 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:27:52.887535 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 622/1227] 10.222.0.15 () {56 vars in 1143 bytes} [Wed Aug 24 05:27:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:27:52.894245 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e0740-53d2-41c1-a701-19bc76416399 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:27:52.998176 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e0740-53d2-41c1-a701-19bc76416399 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:27:53.031448 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e5ab7-df78-4dff-b6fb-9f56d2f55fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:27:53.032018 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e5ab7-df78-4dff-b6fb-9f56d2f55fff None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:53.032292 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e5ab7-df78-4dff-b6fb-9f56d2f55fff None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:53.032843 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e5ab7-df78-4dff-b6fb-9f56d2f55fff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:53.043212 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019e5ab7-df78-4dff-b6fb-9f56d2f55fff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:27:53.044673 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019e5ab7-df78-4dff-b6fb-9f56d2f55fff None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:27:53.058079 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-019e5ab7-df78-4dff-b6fb-9f56d2f55fff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:27:53.058079 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:27:53.058079 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:27:53.058079 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:27:53.058079 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:27:53.058079 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:27:53.058079 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:27:53.058079 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:27:53.058079 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:27:53.058079 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:27:53.058079 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:27:53.058079 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:27:53.058079 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:27:53.058079 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:27:53.058079 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:27:53.058079 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:27:53.059117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:27:53.059117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:27:53.059117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:27:53.059117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:27:53.059117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:27:53.059117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:27:53.059117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:27:53.059117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:27:53.059117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:27:53.059117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:27:53.059117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:27:53.059117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:27:53.059117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:27:53.059117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:27:53.059117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:27:53.059117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:27:53.059117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:27:53.060503 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:27:53.060503 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:27:53.060503 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:27:53.060503 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 606/1228] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:27:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:27:53.065830 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a1c7cf-02a1-4aa1-be8c-0b71a1dcd078 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:27:53.176237 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a1c7cf-02a1-4aa1-be8c-0b71a1dcd078 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:27:53.178631 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:27:53.211070 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e4f11e-2e5c-4536-b8d4-b77bff12ac79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:27:53.211673 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e4f11e-2e5c-4536-b8d4-b77bff12ac79 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:53.211958 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e4f11e-2e5c-4536-b8d4-b77bff12ac79 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:53.212235 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e4f11e-2e5c-4536-b8d4-b77bff12ac79 None admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:53.212908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e4f11e-2e5c-4536-b8d4-b77bff12ac79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:27:53.213217 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e4f11e-2e5c-4536-b8d4-b77bff12ac79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:27:53.214576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e4f11e-2e5c-4536-b8d4-b77bff12ac79 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:27:53.225725 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 623/1229] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:27:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:27:53.231933 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e5ab7-df78-4dff-b6fb-9f56d2f55fff None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:27:53.242510 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:27:53.333079 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e5ab7-df78-4dff-b6fb-9f56d2f55fff None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:27:53.366918 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e54fac9-c3a2-47ac-a103-158f0b1e4cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:27:53.367540 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e54fac9-c3a2-47ac-a103-158f0b1e4cc3 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:53.367820 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e54fac9-c3a2-47ac-a103-158f0b1e4cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:53.368094 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e54fac9-c3a2-47ac-a103-158f0b1e4cc3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:53.374812 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e54fac9-c3a2-47ac-a103-158f0b1e4cc3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:27:53.376248 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e54fac9-c3a2-47ac-a103-158f0b1e4cc3 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:27:53.387750 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-8e54fac9-c3a2-47ac-a103-158f0b1e4cc3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:27:53.387750 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:27:53.387750 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:27:53.387750 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:27:53.387750 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:27:53.387750 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:27:53.387750 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:27:53.387750 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:27:53.387750 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:27:53.387750 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:27:53.387750 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:27:53.387750 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:27:53.387750 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:27:53.387750 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:27:53.387750 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:27:53.388828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:27:53.388828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:27:53.388828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:27:53.388828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:27:53.388828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:27:53.388828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:27:53.388828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:27:53.388828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:27:53.388828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:27:53.388828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:27:53.388828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:27:53.388828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:27:53.388828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:27:53.388828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:27:53.388828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:27:53.388828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:27:53.388828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:27:53.389881 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:27:53.389881 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:27:53.389881 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:27:53.389881 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:27:53.389881 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:27:53.389881 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:27:53.389881 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:27:53.389881 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:27:53.389881 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 607/1230] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:27:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:27:53.395847 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e4f11e-2e5c-4536-b8d4-b77bff12ac79 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:27:53.501767 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e4f11e-2e5c-4536-b8d4-b77bff12ac79 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:27:53.537217 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d3576-7453-478f-a2dc-e416cb87d990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:27:53.537842 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d3576-7453-478f-a2dc-e416cb87d990 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:53.538088 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d3576-7453-478f-a2dc-e416cb87d990 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:53.538753 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d3576-7453-478f-a2dc-e416cb87d990 None admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:53.539447 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d3576-7453-478f-a2dc-e416cb87d990 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:27:53.539873 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d3576-7453-478f-a2dc-e416cb87d990 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:27:53.541351 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d3576-7453-478f-a2dc-e416cb87d990 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:27:53.553760 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 624/1231] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Aug 24 05:27:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:27:53.560191 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e54fac9-c3a2-47ac-a103-158f0b1e4cc3 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:27:53.626623 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:27:53.662056 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e54fac9-c3a2-47ac-a103-158f0b1e4cc3 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:27:53.694594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382c24b5-b24b-46e5-863c-1e3ae4e9b6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:27:53.695238 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c24b5-b24b-46e5-863c-1e3ae4e9b6b2 None admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:53.702821 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c24b5-b24b-46e5-863c-1e3ae4e9b6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:53.702821 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c24b5-b24b-46e5-863c-1e3ae4e9b6b2 None admin] PATH_INFO: `/v3/projects/59592706ae9442e08353158c625e9581/users/6778b7c5ad19484b877494f143050405/roles/cb40cd8499f7418ca0da5ea0f1b87ff9` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:53.709257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382c24b5-b24b-46e5-863c-1e3ae4e9b6b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=59592706ae9442e08353158c625e9581, user_id=6778b7c5ad19484b877494f143050405, role_id=cb40cd8499f7418ca0da5ea0f1b87ff9)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:27:53.710743 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382c24b5-b24b-46e5-863c-1e3ae4e9b6b2 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:27:53.851776 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 608/1232] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Aug 24 05:27:53 2022] PUT /identity/v3/projects/59592706ae9442e08353158c625e9581/users/6778b7c5ad19484b877494f143050405/roles/cb40cd8499f7418ca0da5ea0f1b87ff9 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:27:55.744840 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b11006-8671-43cc-ae28-e0027f3baaf4 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:55.745402 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b11006-8671-43cc-ae28-e0027f3baaf4 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:55.745402 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b11006-8671-43cc-ae28-e0027f3baaf4 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:55.746707 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 625/1233] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:27:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:27:55.751950 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33301d-ee9f-443e-a587-cd92016bd0fd None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:55.752474 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33301d-ee9f-443e-a587-cd92016bd0fd None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:55.752669 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33301d-ee9f-443e-a587-cd92016bd0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:55.773737 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-8b33301d-ee9f-443e-a587-cd92016bd0fd None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:27:55.774919 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-8b33301d-ee9f-443e-a587-cd92016bd0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:27:55.849201 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 609/1234] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:27:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:27:56.118966 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b11006-8671-43cc-ae28-e0027f3baaf4 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:27:56.145488 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:27:56.220417 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b11006-8671-43cc-ae28-e0027f3baaf4 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:27:56.257374 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd1bd2c-7eb6-4458-a853-43772fc39311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:27:56.257995 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1bd2c-7eb6-4458-a853-43772fc39311 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:56.258276 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1bd2c-7eb6-4458-a853-43772fc39311 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:56.258556 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1bd2c-7eb6-4458-a853-43772fc39311 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:56.269007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd1bd2c-7eb6-4458-a853-43772fc39311 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:27:56.273681 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-abd1bd2c-7eb6-4458-a853-43772fc39311 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:27:56.273681 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:27:56.273681 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:27:56.273681 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:27:56.273681 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:27:56.273681 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:27:56.273681 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:27:56.273681 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:27:56.273681 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:27:56.273681 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:27:56.273681 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:27:56.273681 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:27:56.273681 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:27:56.273681 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:27:56.273681 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:27:56.273681 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:27:56.274809 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:27:56.274809 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:27:56.274809 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:27:56.274809 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:27:56.274809 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:27:56.274809 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:27:56.274809 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:27:56.274809 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:27:56.274809 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:27:56.274809 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:27:56.274809 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:27:56.274809 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:27:56.274809 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:27:56.274809 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:27:56.274809 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:27:56.274809 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:27:56.275699 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:27:56.275699 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:27:56.275699 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:27:56.275699 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:27:56.275699 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:27:56.275699 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:27:56.275699 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:27:56.275699 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:27:56.275699 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:27:56.275699 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:27:56.275699 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:27:56.275699 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:27:56.275699 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 626/1235] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Aug 24 05:27:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:27:56.281099 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b33301d-ee9f-443e-a587-cd92016bd0fd None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:27:56.328890 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:27:56.381747 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b33301d-ee9f-443e-a587-cd92016bd0fd None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:27:56.417238 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a55c7f-4e48-4de5-8598-e5569b6b43fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:27:56.417908 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a55c7f-4e48-4de5-8598-e5569b6b43fb None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:56.418193 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a55c7f-4e48-4de5-8598-e5569b6b43fb None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:56.418470 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a55c7f-4e48-4de5-8598-e5569b6b43fb None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:56.419137 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a55c7f-4e48-4de5-8598-e5569b6b43fb None admin] RBAC: Adding query filter params (name=service) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:27:56.419444 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a55c7f-4e48-4de5-8598-e5569b6b43fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:27:56.420934 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a55c7f-4e48-4de5-8598-e5569b6b43fb None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:27:56.429388 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 610/1236] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Aug 24 05:27:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:27:56.435114 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd1bd2c-7eb6-4458-a853-43772fc39311 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:27:56.522158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:27:56.539501 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd1bd2c-7eb6-4458-a853-43772fc39311 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:27:56.575639 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a7497-ad23-4b59-8517-787f4597c464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:27:56.576258 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a7497-ad23-4b59-8517-787f4597c464 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:56.576568 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a7497-ad23-4b59-8517-787f4597c464 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:56.576843 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a7497-ad23-4b59-8517-787f4597c464 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:56.587059 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284a7497-ad23-4b59-8517-787f4597c464 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:27:56.588573 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284a7497-ad23-4b59-8517-787f4597c464 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:27:56.603565 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-284a7497-ad23-4b59-8517-787f4597c464 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:27:56.603565 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:27:56.603565 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:27:56.603565 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:27:56.603565 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:27:56.603565 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:27:56.603565 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:27:56.603565 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:27:56.603565 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:27:56.603565 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:27:56.603565 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:27:56.603565 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:27:56.603565 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:27:56.603565 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:27:56.603565 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:27:56.603565 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:27:56.604785 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:27:56.604785 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:27:56.604785 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:27:56.604785 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:27:56.604785 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:27:56.604785 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:27:56.604785 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:27:56.604785 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:27:56.604785 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:27:56.604785 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:27:56.604785 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:27:56.604785 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:27:56.604785 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:27:56.604785 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:27:56.604785 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:27:56.604785 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:27:56.604785 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:27:56.605760 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:27:56.605760 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:27:56.605760 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:27:56.605760 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 627/1237] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:27:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:27:56.611050 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a55c7f-4e48-4de5-8598-e5569b6b43fb None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:27:56.716217 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a55c7f-4e48-4de5-8598-e5569b6b43fb None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:27:56.725607 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:27:56.749719 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1453c6d7-3155-4cdd-8a79-324f72f2a06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:27:56.750350 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1453c6d7-3155-4cdd-8a79-324f72f2a06e None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:56.750601 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1453c6d7-3155-4cdd-8a79-324f72f2a06e None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:56.750874 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1453c6d7-3155-4cdd-8a79-324f72f2a06e None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:56.751511 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1453c6d7-3155-4cdd-8a79-324f72f2a06e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:27:56.751816 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1453c6d7-3155-4cdd-8a79-324f72f2a06e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:27:56.753256 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1453c6d7-3155-4cdd-8a79-324f72f2a06e None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:27:56.764661 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 611/1238] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:27:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:27:56.770623 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a7497-ad23-4b59-8517-787f4597c464 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:27:56.874349 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a7497-ad23-4b59-8517-787f4597c464 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:27:56.907067 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df474880-fe11-40c7-b87f-4fd3177922b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:27:56.907963 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df474880-fe11-40c7-b87f-4fd3177922b4 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:56.908244 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df474880-fe11-40c7-b87f-4fd3177922b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:56.909079 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df474880-fe11-40c7-b87f-4fd3177922b4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:56.911810 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:27:56.916657 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df474880-fe11-40c7-b87f-4fd3177922b4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:27:56.918088 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df474880-fe11-40c7-b87f-4fd3177922b4 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:27:56.929889 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-df474880-fe11-40c7-b87f-4fd3177922b4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 05:27:56.929889 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:27:56.929889 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:27:56.929889 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:27:56.929889 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:27:56.929889 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:27:56.929889 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:27:56.929889 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:27:56.929889 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:27:56.929889 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:27:56.929889 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:27:56.929889 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:27:56.929889 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:27:56.929889 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:27:56.929889 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:27:56.929889 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:27:56.931007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:27:56.931007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:27:56.931007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:27:56.931007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:27:56.931007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:27:56.931007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:27:56.931007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:27:56.931007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:27:56.931007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:27:56.931007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:27:56.931007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:27:56.931007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:27:56.931007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:27:56.931007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:27:56.931007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:27:56.931007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:27:56.931007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:27:56.931007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:27:56.932274 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:27:56.932274 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:27:56.932274 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:27:56.932274 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:27:56.932274 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:27:56.932274 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:27:56.932274 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:27:56.932274 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:27:56.932274 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 05:27:56.932274 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:27:56.932274 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 628/1239] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:27:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:27:56.937160 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1453c6d7-3155-4cdd-8a79-324f72f2a06e None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:27:57.040038 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1453c6d7-3155-4cdd-8a79-324f72f2a06e None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:27:57.074557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4a1218-c493-4c69-befa-1021bd1522ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:27:57.075160 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4a1218-c493-4c69-befa-1021bd1522ee None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:57.075444 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4a1218-c493-4c69-befa-1021bd1522ee None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:57.075722 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4a1218-c493-4c69-befa-1021bd1522ee None admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:57.076470 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4a1218-c493-4c69-befa-1021bd1522ee None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:27:57.076784 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4a1218-c493-4c69-befa-1021bd1522ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:27:57.079092 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4a1218-c493-4c69-befa-1021bd1522ee None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:27:57.098111 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 612/1240] 10.222.0.15 () {56 vars in 1143 bytes} [Wed Aug 24 05:27:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:27:57.103900 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df474880-fe11-40c7-b87f-4fd3177922b4 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:27:57.203950 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df474880-fe11-40c7-b87f-4fd3177922b4 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:27:57.238438 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2101e288-e072-4ac8-8d28-edec7a34c219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:27:57.239044 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101e288-e072-4ac8-8d28-edec7a34c219 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:57.239330 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101e288-e072-4ac8-8d28-edec7a34c219 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:57.239607 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101e288-e072-4ac8-8d28-edec7a34c219 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:57.300050 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:27:57.306182 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2101e288-e072-4ac8-8d28-edec7a34c219 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:27:57.307635 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2101e288-e072-4ac8-8d28-edec7a34c219 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:27:57.408084 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 629/1241] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:27:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:27:57.414409 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4a1218-c493-4c69-befa-1021bd1522ee None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:27:57.442149 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:27:57.514565 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4a1218-c493-4c69-befa-1021bd1522ee None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:27:57.552560 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec8e4c4-dd3d-4755-9022-9b83aef13a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:27:57.553158 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8e4c4-dd3d-4755-9022-9b83aef13a0b None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:57.553680 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8e4c4-dd3d-4755-9022-9b83aef13a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:57.553997 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8e4c4-dd3d-4755-9022-9b83aef13a0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:57.565110 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec8e4c4-dd3d-4755-9022-9b83aef13a0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:27:57.566732 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec8e4c4-dd3d-4755-9022-9b83aef13a0b None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:27:57.581302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-0ec8e4c4-dd3d-4755-9022-9b83aef13a0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:27:57.581302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:27:57.581302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:27:57.581302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:27:57.581302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:27:57.581302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:27:57.581302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:27:57.581302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:27:57.581302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:27:57.581302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:27:57.581302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:27:57.581302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:27:57.581302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:27:57.581302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:27:57.581302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:27:57.581302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:27:57.582263 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:27:57.582263 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:27:57.582263 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:27:57.582263 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:27:57.582263 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:27:57.582263 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:27:57.582263 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:27:57.582263 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:27:57.582263 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:27:57.582263 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:27:57.582263 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:27:57.582263 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:27:57.582263 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:27:57.582263 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:27:57.582263 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:27:57.582263 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:27:57.582263 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:27:57.583245 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:27:57.583245 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:27:57.583245 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:27:57.583245 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 613/1242] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:27:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:27:57.588348 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2101e288-e072-4ac8-8d28-edec7a34c219 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:27:57.680931 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:27:57.686514 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2101e288-e072-4ac8-8d28-edec7a34c219 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:27:57.719866 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65a86ba-ca7b-46ef-a1e0-5fe1640b89fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:27:57.720498 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a86ba-ca7b-46ef-a1e0-5fe1640b89fb None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:57.720776 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a86ba-ca7b-46ef-a1e0-5fe1640b89fb None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:57.721049 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a86ba-ca7b-46ef-a1e0-5fe1640b89fb None admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:57.721712 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65a86ba-ca7b-46ef-a1e0-5fe1640b89fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:27:57.722040 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65a86ba-ca7b-46ef-a1e0-5fe1640b89fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:27:57.723594 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65a86ba-ca7b-46ef-a1e0-5fe1640b89fb None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:27:57.734997 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 630/1243] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:27:57.741447 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec8e4c4-dd3d-4755-9022-9b83aef13a0b None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:27:57.820440 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:27:57.841071 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec8e4c4-dd3d-4755-9022-9b83aef13a0b None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:27:57.874163 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776e9e0-b769-4e92-85b4-c79b82b86970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:27:57.874785 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776e9e0-b769-4e92-85b4-c79b82b86970 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:57.875063 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776e9e0-b769-4e92-85b4-c79b82b86970 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:57.875333 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776e9e0-b769-4e92-85b4-c79b82b86970 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:57.882520 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3776e9e0-b769-4e92-85b4-c79b82b86970 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:27:57.883952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3776e9e0-b769-4e92-85b4-c79b82b86970 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:27:57.894731 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-3776e9e0-b769-4e92-85b4-c79b82b86970 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:27:57.894731 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:27:57.894731 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:27:57.894731 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:27:57.894731 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:27:57.894731 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:27:57.894731 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:27:57.894731 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:27:57.894731 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:27:57.894731 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:27:57.894731 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:27:57.894731 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:27:57.894731 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:27:57.894731 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:27:57.894731 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:27:57.895798 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:27:57.895798 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:27:57.895798 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:27:57.895798 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:27:57.895798 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:27:57.895798 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:27:57.895798 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:27:57.895798 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:27:57.895798 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:27:57.895798 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:27:57.895798 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:27:57.895798 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:27:57.895798 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:27:57.895798 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:27:57.895798 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:27:57.895798 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:27:57.895798 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:27:57.897140 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:27:57.897140 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:27:57.897140 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:27:57.897140 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:27:57.897140 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:27:57.897140 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:27:57.897140 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:27:57.897140 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:27:57.897140 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 614/1244] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:27:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:27:57.901749 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65a86ba-ca7b-46ef-a1e0-5fe1640b89fb None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:27:58.005480 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65a86ba-ca7b-46ef-a1e0-5fe1640b89fb None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:27:58.037878 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cdcd7c-05a6-4c52-b27b-876b7c710695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:27:58.038475 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cdcd7c-05a6-4c52-b27b-876b7c710695 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:58.038755 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cdcd7c-05a6-4c52-b27b-876b7c710695 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:58.039030 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cdcd7c-05a6-4c52-b27b-876b7c710695 None admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:58.039711 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cdcd7c-05a6-4c52-b27b-876b7c710695 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:27:58.040059 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cdcd7c-05a6-4c52-b27b-876b7c710695 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:27:58.041521 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cdcd7c-05a6-4c52-b27b-876b7c710695 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:27:58.053582 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 631/1245] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Aug 24 05:27:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:27:58.059719 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776e9e0-b769-4e92-85b4-c79b82b86970 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:27:58.161437 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776e9e0-b769-4e92-85b4-c79b82b86970 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:27:58.196466 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd7e63-8876-461c-b8b6-9cf894b4b8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:27:58.197076 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd7e63-8876-461c-b8b6-9cf894b4b8d5 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:58.197396 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd7e63-8876-461c-b8b6-9cf894b4b8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:58.197667 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd7e63-8876-461c-b8b6-9cf894b4b8d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:58.198385 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bd7e63-8876-461c-b8b6-9cf894b4b8d5 None admin] RBAC: Adding query filter params (role.id=cb40cd8499f7418ca0da5ea0f1b87ff9, scope.project.id=59592706ae9442e08353158c625e9581, user.id=6778b7c5ad19484b877494f143050405) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:27:58.198691 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bd7e63-8876-461c-b8b6-9cf894b4b8d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:27:58.200113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bd7e63-8876-461c-b8b6-9cf894b4b8d5 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:27:58.205730 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 615/1246] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:27:58 2022] GET /identity/v3/role_assignments?user.id=6778b7c5ad19484b877494f143050405&scope.project.id=59592706ae9442e08353158c625e9581&role.id=cb40cd8499f7418ca0da5ea0f1b87ff9 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:28:00.267787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0419f43-7419-4416-a107-a0db5cfc7054 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:00.268359 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0419f43-7419-4416-a107-a0db5cfc7054 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:00.268359 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0419f43-7419-4416-a107-a0db5cfc7054 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:00.270038 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 632/1247] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:28:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:28:00.275447 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802af0c-75d3-46f9-a9b6-d51e4382746f None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:00.278431 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802af0c-75d3-46f9-a9b6-d51e4382746f None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:00.278431 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802af0c-75d3-46f9-a9b6-d51e4382746f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:00.282716 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:28:00.294900 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-0802af0c-75d3-46f9-a9b6-d51e4382746f None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:28:00.296098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-0802af0c-75d3-46f9-a9b6-d51e4382746f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:28:00.363708 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 616/1248] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:28:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:28:00.644317 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0419f43-7419-4416-a107-a0db5cfc7054 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:00.667233 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:28:00.752180 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0419f43-7419-4416-a107-a0db5cfc7054 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:00.788573 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d6626c-2c37-49df-b847-da40f455f0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:00.789212 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d6626c-2c37-49df-b847-da40f455f0c7 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:00.789496 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d6626c-2c37-49df-b847-da40f455f0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:00.789777 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d6626c-2c37-49df-b847-da40f455f0c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:00.800135 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d6626c-2c37-49df-b847-da40f455f0c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:28:00.805063 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-47d6626c-2c37-49df-b847-da40f455f0c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:28:00.805063 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:28:00.805063 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:28:00.805063 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:28:00.805063 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:28:00.805063 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:28:00.805063 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:28:00.805063 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:28:00.805063 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:28:00.805063 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:28:00.805063 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:28:00.805063 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:28:00.805063 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:28:00.805063 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:28:00.805063 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:28:00.805063 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:28:00.806229 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:28:00.806229 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:28:00.806229 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:28:00.806229 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:28:00.806229 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:28:00.806229 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:28:00.806229 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:28:00.806229 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:28:00.806229 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:28:00.806229 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:28:00.806229 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:28:00.806229 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:28:00.806229 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:28:00.806229 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:28:00.806229 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:28:00.806229 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:28:00.807458 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:28:00.807458 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:28:00.807458 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:28:00.807458 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:28:00.807458 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:28:00.807458 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:28:00.807458 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:28:00.807458 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:28:00.807458 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:28:00.807458 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:28:00.807458 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:28:00.807458 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:28:00.807458 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 633/1249] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Aug 24 05:28:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:28:00.812231 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802af0c-75d3-46f9-a9b6-d51e4382746f None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:00.911926 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802af0c-75d3-46f9-a9b6-d51e4382746f None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:00.940548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:28:00.945417 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cb00e3-4509-4400-875d-85041fdfbec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:00.946023 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb00e3-4509-4400-875d-85041fdfbec5 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:00.946307 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb00e3-4509-4400-875d-85041fdfbec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:00.946585 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb00e3-4509-4400-875d-85041fdfbec5 None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:00.947251 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cb00e3-4509-4400-875d-85041fdfbec5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:28:00.947556 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cb00e3-4509-4400-875d-85041fdfbec5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:00.949036 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cb00e3-4509-4400-875d-85041fdfbec5 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:00.957528 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 617/1250] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:28:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:28:00.963530 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d6626c-2c37-49df-b847-da40f455f0c7 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:01.041881 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:28:01.063692 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d6626c-2c37-49df-b847-da40f455f0c7 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:01.097596 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da7629-3995-4674-bca8-8b70cd8783c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:01.104081 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da7629-3995-4674-bca8-8b70cd8783c1 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:01.104081 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da7629-3995-4674-bca8-8b70cd8783c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:01.104081 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da7629-3995-4674-bca8-8b70cd8783c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:01.109552 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88da7629-3995-4674-bca8-8b70cd8783c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:01.111209 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88da7629-3995-4674-bca8-8b70cd8783c1 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:01.138863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-88da7629-3995-4674-bca8-8b70cd8783c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:28:01.138863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:28:01.138863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:28:01.138863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:28:01.138863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:28:01.138863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:28:01.138863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:28:01.138863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:28:01.138863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:28:01.138863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:28:01.138863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:28:01.138863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:28:01.138863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:28:01.138863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:28:01.138863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:28:01.138863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:28:01.145428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:28:01.145428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:28:01.145428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:28:01.145428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:28:01.145428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:28:01.145428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:28:01.145428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:28:01.145428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:28:01.145428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:28:01.145428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:28:01.145428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:28:01.145428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:28:01.145428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:28:01.145428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:28:01.145428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:28:01.145428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:28:01.145428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:28:01.146506 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:28:01.146506 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:28:01.146506 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:28:01.146506 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 634/1251] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:28:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:28:01.146910 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cb00e3-4509-4400-875d-85041fdfbec5 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:01.247239 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cb00e3-4509-4400-875d-85041fdfbec5 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:01.281214 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b0506-e499-4f83-9cd3-7cd42cd3bcb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:01.281821 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b0506-e499-4f83-9cd3-7cd42cd3bcb0 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:01.282087 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b0506-e499-4f83-9cd3-7cd42cd3bcb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:01.282361 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b0506-e499-4f83-9cd3-7cd42cd3bcb0 None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:01.283303 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4b0506-e499-4f83-9cd3-7cd42cd3bcb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:28:01.283618 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4b0506-e499-4f83-9cd3-7cd42cd3bcb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:01.285168 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4b0506-e499-4f83-9cd3-7cd42cd3bcb0 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:01.296528 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 618/1252] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:28:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:28:01.302296 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da7629-3995-4674-bca8-8b70cd8783c1 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:01.400090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da7629-3995-4674-bca8-8b70cd8783c1 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:01.427619 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:28:01.444442 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897b864-9306-4e47-81aa-9974d962c2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:01.445027 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897b864-9306-4e47-81aa-9974d962c2be None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:01.445310 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897b864-9306-4e47-81aa-9974d962c2be None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:01.445586 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897b864-9306-4e47-81aa-9974d962c2be None admin] PATH_INFO: `/v3/users/placement` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:01.453340 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4897b864-9306-4e47-81aa-9974d962c2be None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:01.454819 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4897b864-9306-4e47-81aa-9974d962c2be None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:01.467898 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-4897b864-9306-4e47-81aa-9974d962c2be None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 05:28:01.467898 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:28:01.467898 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:28:01.467898 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:28:01.467898 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:28:01.467898 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:28:01.467898 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:28:01.467898 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:28:01.467898 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:28:01.467898 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:28:01.467898 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:28:01.467898 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:28:01.467898 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:28:01.467898 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:28:01.467898 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:28:01.467898 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:28:01.469126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:28:01.469126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:28:01.469126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:28:01.469126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:28:01.469126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:28:01.469126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:28:01.469126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:28:01.469126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:28:01.469126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:28:01.469126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:28:01.469126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:28:01.469126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:28:01.469126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:28:01.469126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:28:01.469126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:28:01.469126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:28:01.469126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:28:01.469126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:28:01.470129 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:28:01.470129 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:28:01.470129 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:28:01.470129 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:28:01.470129 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:28:01.470129 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:28:01.470129 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:28:01.470129 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:28:01.470129 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 05:28:01.470129 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:28:01.470129 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 635/1253] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:28:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:28:01.474881 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b0506-e499-4f83-9cd3-7cd42cd3bcb0 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:01.532092 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:28:01.577198 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b0506-e499-4f83-9cd3-7cd42cd3bcb0 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:01.610097 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c2b14-2a7c-4dd9-b043-2b58f9ad6841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:01.610696 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c2b14-2a7c-4dd9-b043-2b58f9ad6841 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:01.610974 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c2b14-2a7c-4dd9-b043-2b58f9ad6841 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:01.611687 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c2b14-2a7c-4dd9-b043-2b58f9ad6841 None admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:01.611979 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92c2b14-2a7c-4dd9-b043-2b58f9ad6841 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:28:01.612284 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92c2b14-2a7c-4dd9-b043-2b58f9ad6841 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:01.614298 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92c2b14-2a7c-4dd9-b043-2b58f9ad6841 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:01.631833 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 619/1254] 10.222.0.15 () {56 vars in 1143 bytes} [Wed Aug 24 05:28:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:28:01.643800 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897b864-9306-4e47-81aa-9974d962c2be None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:01.737101 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897b864-9306-4e47-81aa-9974d962c2be None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:01.775384 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9251957e-856b-45b2-82d4-5c98ef073ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:01.775997 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251957e-856b-45b2-82d4-5c98ef073ad1 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:01.776284 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251957e-856b-45b2-82d4-5c98ef073ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:01.776594 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251957e-856b-45b2-82d4-5c98ef073ad1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:01.833173 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:28:01.845076 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9251957e-856b-45b2-82d4-5c98ef073ad1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:01.846662 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9251957e-856b-45b2-82d4-5c98ef073ad1 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:01.948469 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 636/1255] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:28:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:28:01.954297 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c2b14-2a7c-4dd9-b043-2b58f9ad6841 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:02.052295 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c2b14-2a7c-4dd9-b043-2b58f9ad6841 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:02.065178 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:28:02.085455 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fd4f78-6199-4c70-af3e-d15a1a478b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:02.086067 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd4f78-6199-4c70-af3e-d15a1a478b2a None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:02.086347 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd4f78-6199-4c70-af3e-d15a1a478b2a None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:02.086616 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd4f78-6199-4c70-af3e-d15a1a478b2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:02.097262 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fd4f78-6199-4c70-af3e-d15a1a478b2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:02.098932 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fd4f78-6199-4c70-af3e-d15a1a478b2a None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:02.113870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-21fd4f78-6199-4c70-af3e-d15a1a478b2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:28:02.113870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:28:02.113870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:28:02.113870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:28:02.113870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:28:02.113870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:28:02.113870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:28:02.113870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:28:02.113870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:28:02.113870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:28:02.113870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:28:02.113870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:28:02.113870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:28:02.113870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:28:02.113870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:28:02.113870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:28:02.114992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:28:02.114992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:28:02.114992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:28:02.114992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:28:02.114992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:28:02.114992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:28:02.114992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:28:02.114992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:28:02.114992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:28:02.114992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:28:02.114992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:28:02.114992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:28:02.114992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:28:02.114992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:28:02.114992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:28:02.114992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:28:02.114992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:28:02.116189 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:28:02.116189 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:28:02.116189 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:28:02.116189 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 620/1256] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:28:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:28:02.120856 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9251957e-856b-45b2-82d4-5c98ef073ad1 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:02.207167 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:28:02.218515 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9251957e-856b-45b2-82d4-5c98ef073ad1 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:02.251861 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee8f9e-2b74-4043-a9f2-4a5a30df0d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:02.252563 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee8f9e-2b74-4043-a9f2-4a5a30df0d45 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:02.252864 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee8f9e-2b74-4043-a9f2-4a5a30df0d45 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:02.253174 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee8f9e-2b74-4043-a9f2-4a5a30df0d45 None admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:02.253859 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee8f9e-2b74-4043-a9f2-4a5a30df0d45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:28:02.254163 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee8f9e-2b74-4043-a9f2-4a5a30df0d45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:02.255519 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee8f9e-2b74-4043-a9f2-4a5a30df0d45 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:02.266467 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 637/1257] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:28:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:28:02.272764 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fd4f78-6199-4c70-af3e-d15a1a478b2a None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:02.376073 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fd4f78-6199-4c70-af3e-d15a1a478b2a None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:02.410822 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e6f97-342c-4f3a-acbd-ba359aa964b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:02.411446 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e6f97-342c-4f3a-acbd-ba359aa964b3 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:02.411769 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e6f97-342c-4f3a-acbd-ba359aa964b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:02.412047 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e6f97-342c-4f3a-acbd-ba359aa964b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:02.418580 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e6f97-342c-4f3a-acbd-ba359aa964b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:02.420055 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e6f97-342c-4f3a-acbd-ba359aa964b3 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:02.431614 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-fd8e6f97-342c-4f3a-acbd-ba359aa964b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:28:02.431614 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:28:02.431614 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:28:02.431614 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:28:02.431614 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:28:02.431614 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:28:02.431614 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:28:02.431614 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:28:02.431614 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:28:02.431614 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:28:02.431614 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:28:02.431614 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:28:02.431614 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:28:02.431614 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:28:02.431614 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:28:02.432736 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:28:02.432736 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:28:02.432736 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:28:02.432736 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:28:02.432736 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:28:02.432736 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:28:02.432736 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:28:02.432736 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:28:02.432736 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:28:02.432736 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:28:02.432736 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:28:02.432736 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:28:02.432736 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:28:02.432736 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:28:02.432736 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:28:02.432736 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:28:02.432736 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:28:02.433747 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:28:02.433747 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:28:02.433747 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:28:02.433747 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:28:02.433747 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:28:02.433747 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:28:02.433747 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:28:02.433747 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:28:02.433747 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 621/1258] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:28:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:28:02.438727 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee8f9e-2b74-4043-a9f2-4a5a30df0d45 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:02.539811 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee8f9e-2b74-4043-a9f2-4a5a30df0d45 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:02.575879 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550de540-111a-4c3f-a02c-6f9306679a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:02.576520 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550de540-111a-4c3f-a02c-6f9306679a4c None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:02.576806 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550de540-111a-4c3f-a02c-6f9306679a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:02.577085 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550de540-111a-4c3f-a02c-6f9306679a4c None admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:02.577780 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550de540-111a-4c3f-a02c-6f9306679a4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:28:02.578090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550de540-111a-4c3f-a02c-6f9306679a4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:02.579577 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550de540-111a-4c3f-a02c-6f9306679a4c None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:02.592652 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 638/1259] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Aug 24 05:28:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:28:02.600270 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e6f97-342c-4f3a-acbd-ba359aa964b3 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:02.630592 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:28:02.709209 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e6f97-342c-4f3a-acbd-ba359aa964b3 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:02.746255 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6820de1-db27-479d-8190-739ad6bc1306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:02.746859 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6820de1-db27-479d-8190-739ad6bc1306 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:02.747143 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6820de1-db27-479d-8190-739ad6bc1306 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:02.747421 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6820de1-db27-479d-8190-739ad6bc1306 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:02.748157 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6820de1-db27-479d-8190-739ad6bc1306 None admin] RBAC: Adding query filter params (role.id=e013ad4d436f4cb0a9bf7debb7d13dc7, scope.project.id=59592706ae9442e08353158c625e9581, user.id=6778b7c5ad19484b877494f143050405) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:28:02.748506 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6820de1-db27-479d-8190-739ad6bc1306 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:02.749991 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6820de1-db27-479d-8190-739ad6bc1306 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:02.756313 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 622/1260] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:28:02 2022] GET /identity/v3/role_assignments?user.id=6778b7c5ad19484b877494f143050405&scope.project.id=59592706ae9442e08353158c625e9581&role.id=e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:28:04.664867 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5c265-cd6b-43c5-b96c-466f0f9394be None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:04.665415 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5c265-cd6b-43c5-b96c-466f0f9394be None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:04.665415 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5c265-cd6b-43c5-b96c-466f0f9394be None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:04.666750 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 639/1261] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:28:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:28:04.671923 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b251e2-20ac-4b16-8cad-10314e0e4b5b None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:04.672308 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b251e2-20ac-4b16-8cad-10314e0e4b5b None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:04.672512 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b251e2-20ac-4b16-8cad-10314e0e4b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:04.697069 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-99b251e2-20ac-4b16-8cad-10314e0e4b5b None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:28:04.698272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-99b251e2-20ac-4b16-8cad-10314e0e4b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:28:04.765529 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 623/1262] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:28:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:28:05.044939 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac5c265-cd6b-43c5-b96c-466f0f9394be None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:05.056716 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:28:05.158222 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac5c265-cd6b-43c5-b96c-466f0f9394be None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:05.194362 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f7e21-b581-4485-9add-7f34b41df1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:05.194997 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f7e21-b581-4485-9add-7f34b41df1be None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:05.195296 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f7e21-b581-4485-9add-7f34b41df1be None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:05.195554 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f7e21-b581-4485-9add-7f34b41df1be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:05.205440 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f7e21-b581-4485-9add-7f34b41df1be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:28:05.210576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-716f7e21-b581-4485-9add-7f34b41df1be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:28:05.210576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:28:05.210576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:28:05.210576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:28:05.210576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:28:05.210576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:28:05.210576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:28:05.210576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:28:05.210576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:28:05.210576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:28:05.210576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:28:05.210576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:28:05.210576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:28:05.210576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:28:05.210576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:28:05.210576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:28:05.212567 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:28:05.212567 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:28:05.212567 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:28:05.212567 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:28:05.212567 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:28:05.212567 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:28:05.212567 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:28:05.212567 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:28:05.212567 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:28:05.212567 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:28:05.212567 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:28:05.212567 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:28:05.212567 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:28:05.212567 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:28:05.212567 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:28:05.212567 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:28:05.213439 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:28:05.213439 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:28:05.213439 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:28:05.213439 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:28:05.213439 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:28:05.213439 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:28:05.213439 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:28:05.213439 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:28:05.213439 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:28:05.213439 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:28:05.213439 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:28:05.213439 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:28:05.213439 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 640/1263] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Aug 24 05:28:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:28:05.217729 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b251e2-20ac-4b16-8cad-10314e0e4b5b None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:05.236088 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:28:05.317707 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b251e2-20ac-4b16-8cad-10314e0e4b5b None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:05.353132 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae8e3a1-7a66-49e7-8eb9-871650ce3615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:05.353727 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8e3a1-7a66-49e7-8eb9-871650ce3615 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:05.354008 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8e3a1-7a66-49e7-8eb9-871650ce3615 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:05.354399 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8e3a1-7a66-49e7-8eb9-871650ce3615 None admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:05.355097 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae8e3a1-7a66-49e7-8eb9-871650ce3615 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:28:05.355406 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae8e3a1-7a66-49e7-8eb9-871650ce3615 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:05.356797 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae8e3a1-7a66-49e7-8eb9-871650ce3615 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:05.364811 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 624/1264] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:28:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:28:05.370747 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f7e21-b581-4485-9add-7f34b41df1be None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:05.443409 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:28:05.481093 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f7e21-b581-4485-9add-7f34b41df1be None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:05.514643 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5a1de3-98e9-4006-9f92-f9399f0c9503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:05.515295 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a1de3-98e9-4006-9f92-f9399f0c9503 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:05.515533 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a1de3-98e9-4006-9f92-f9399f0c9503 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:05.515806 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a1de3-98e9-4006-9f92-f9399f0c9503 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:05.526606 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5a1de3-98e9-4006-9f92-f9399f0c9503 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:05.528033 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5a1de3-98e9-4006-9f92-f9399f0c9503 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:05.542865 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-8e5a1de3-98e9-4006-9f92-f9399f0c9503 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:28:05.542865 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:28:05.542865 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:28:05.542865 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:28:05.542865 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:28:05.542865 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:28:05.542865 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:28:05.542865 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:28:05.542865 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:28:05.542865 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:28:05.542865 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:28:05.542865 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:28:05.542865 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:28:05.542865 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:28:05.542865 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:28:05.542865 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:28:05.548655 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:28:05.548655 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:28:05.548655 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:28:05.548655 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:28:05.548655 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:28:05.548655 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:28:05.548655 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:28:05.548655 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:28:05.548655 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:28:05.548655 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:28:05.548655 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:28:05.548655 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:28:05.548655 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:28:05.548655 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:28:05.548655 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:28:05.548655 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:28:05.548655 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:28:05.549875 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:28:05.549875 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:28:05.549875 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:28:05.549875 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 641/1265] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:28:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:28:05.550418 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae8e3a1-7a66-49e7-8eb9-871650ce3615 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:05.631712 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:28:05.650519 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae8e3a1-7a66-49e7-8eb9-871650ce3615 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:05.685037 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925e20c-9bb2-4660-9f28-27f3bbe32119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:05.685704 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925e20c-9bb2-4660-9f28-27f3bbe32119 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:05.685911 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925e20c-9bb2-4660-9f28-27f3bbe32119 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:05.686183 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925e20c-9bb2-4660-9f28-27f3bbe32119 None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:05.686829 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c925e20c-9bb2-4660-9f28-27f3bbe32119 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:28:05.687133 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c925e20c-9bb2-4660-9f28-27f3bbe32119 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:05.688519 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c925e20c-9bb2-4660-9f28-27f3bbe32119 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:05.700271 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 625/1266] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:28:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:28:05.706723 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5a1de3-98e9-4006-9f92-f9399f0c9503 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:05.816627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5a1de3-98e9-4006-9f92-f9399f0c9503 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:05.836225 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:28:05.849984 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304b3167-42e0-450e-800e-5b958d68254a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:05.850616 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304b3167-42e0-450e-800e-5b958d68254a None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:05.850901 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304b3167-42e0-450e-800e-5b958d68254a None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:05.851201 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304b3167-42e0-450e-800e-5b958d68254a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:05.858590 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304b3167-42e0-450e-800e-5b958d68254a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:05.860030 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304b3167-42e0-450e-800e-5b958d68254a None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:05.871145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-304b3167-42e0-450e-800e-5b958d68254a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 05:28:05.871145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:28:05.871145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:28:05.871145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:28:05.871145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:28:05.871145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:28:05.871145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:28:05.871145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:28:05.871145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:28:05.871145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:28:05.871145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:28:05.871145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:28:05.871145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:28:05.871145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:28:05.871145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:28:05.871145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:28:05.872330 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:28:05.872330 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:28:05.872330 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:28:05.872330 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:28:05.872330 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:28:05.872330 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:28:05.872330 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:28:05.872330 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:28:05.872330 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:28:05.872330 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:28:05.872330 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:28:05.872330 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:28:05.872330 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:28:05.872330 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:28:05.872330 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:28:05.872330 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:28:05.872330 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:28:05.872330 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:28:05.873559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:28:05.873559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:28:05.873559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:28:05.873559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:28:05.873559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:28:05.873559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:28:05.873559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:28:05.873559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:28:05.873559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 05:28:05.873559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:28:05.873559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 642/1267] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:28:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:28:05.878633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925e20c-9bb2-4660-9f28-27f3bbe32119 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:05.979975 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925e20c-9bb2-4660-9f28-27f3bbe32119 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:06.014382 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e583877-38e4-4517-824a-fbf2c66ae1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:06.014987 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e583877-38e4-4517-824a-fbf2c66ae1e6 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:06.015270 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e583877-38e4-4517-824a-fbf2c66ae1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:06.015544 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e583877-38e4-4517-824a-fbf2c66ae1e6 None admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:06.016269 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e583877-38e4-4517-824a-fbf2c66ae1e6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:28:06.016612 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e583877-38e4-4517-824a-fbf2c66ae1e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:06.018032 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e583877-38e4-4517-824a-fbf2c66ae1e6 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:06.027158 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:28:06.035863 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 626/1268] 10.222.0.15 () {56 vars in 1143 bytes} [Wed Aug 24 05:28:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:28:06.041838 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304b3167-42e0-450e-800e-5b958d68254a None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:06.141783 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304b3167-42e0-450e-800e-5b958d68254a None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:06.175162 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee05a67c-3397-48b7-9951-3bc7870b4892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:06.175786 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee05a67c-3397-48b7-9951-3bc7870b4892 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:06.176065 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee05a67c-3397-48b7-9951-3bc7870b4892 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:06.176339 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee05a67c-3397-48b7-9951-3bc7870b4892 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:06.187896 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee05a67c-3397-48b7-9951-3bc7870b4892 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:06.189436 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee05a67c-3397-48b7-9951-3bc7870b4892 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:06.204061 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-ee05a67c-3397-48b7-9951-3bc7870b4892 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:28:06.204061 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:28:06.204061 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:28:06.204061 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:28:06.204061 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:28:06.204061 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:28:06.204061 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:28:06.204061 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:28:06.204061 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:28:06.204061 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:28:06.204061 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:28:06.204061 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:28:06.204061 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:28:06.204061 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:28:06.204061 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:28:06.204061 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:28:06.205222 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:28:06.205222 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:28:06.205222 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:28:06.205222 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:28:06.205222 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:28:06.205222 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:28:06.205222 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:28:06.205222 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:28:06.205222 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:28:06.205222 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:28:06.205222 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:28:06.205222 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:28:06.205222 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:28:06.205222 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:28:06.205222 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:28:06.205222 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:28:06.205222 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:28:06.206759 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:28:06.206759 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:28:06.206759 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:28:06.206759 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 643/1269] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:28:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:28:06.211677 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e583877-38e4-4517-824a-fbf2c66ae1e6 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:06.311941 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e583877-38e4-4517-824a-fbf2c66ae1e6 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:06.345548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6436b882-5af5-42a9-bcc0-3e368bd5aa91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:06.346178 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436b882-5af5-42a9-bcc0-3e368bd5aa91 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:06.346464 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436b882-5af5-42a9-bcc0-3e368bd5aa91 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:06.347494 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436b882-5af5-42a9-bcc0-3e368bd5aa91 None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:06.348353 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6436b882-5af5-42a9-bcc0-3e368bd5aa91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:28:06.348704 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6436b882-5af5-42a9-bcc0-3e368bd5aa91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:06.350074 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6436b882-5af5-42a9-bcc0-3e368bd5aa91 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:06.361265 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 627/1270] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:28:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:28:06.368015 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee05a67c-3397-48b7-9951-3bc7870b4892 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:06.399064 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:28:06.466153 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee05a67c-3397-48b7-9951-3bc7870b4892 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:06.500973 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f9677-d853-497f-b784-df6fa27428c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:06.501593 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f9677-d853-497f-b784-df6fa27428c3 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:06.501868 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f9677-d853-497f-b784-df6fa27428c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:06.502134 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f9677-d853-497f-b784-df6fa27428c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:06.508665 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595f9677-d853-497f-b784-df6fa27428c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:06.510088 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595f9677-d853-497f-b784-df6fa27428c3 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:06.520993 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-595f9677-d853-497f-b784-df6fa27428c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:28:06.520993 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:28:06.520993 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:28:06.520993 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:28:06.520993 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:28:06.520993 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:28:06.520993 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:28:06.520993 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:28:06.520993 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:28:06.520993 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:28:06.520993 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:28:06.520993 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:28:06.520993 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:28:06.520993 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:28:06.520993 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:28:06.527611 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:28:06.527611 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:28:06.527611 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:28:06.527611 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:28:06.527611 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:28:06.527611 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:28:06.527611 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:28:06.527611 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:28:06.527611 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:28:06.527611 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:28:06.527611 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:28:06.527611 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:28:06.527611 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:28:06.527611 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:28:06.527611 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:28:06.527611 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:28:06.527611 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:28:06.528702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:28:06.528702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:28:06.528702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:28:06.528702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:28:06.528702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:28:06.528702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:28:06.528702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:28:06.528702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:28:06.528702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 644/1271] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:28:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:28:06.529283 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6436b882-5af5-42a9-bcc0-3e368bd5aa91 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:06.589187 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:28:06.633807 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6436b882-5af5-42a9-bcc0-3e368bd5aa91 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:06.667302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7bf0f7-0076-4d76-9561-c6e8b4ac2355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:06.670913 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7bf0f7-0076-4d76-9561-c6e8b4ac2355 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:06.670913 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7bf0f7-0076-4d76-9561-c6e8b4ac2355 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:06.670913 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7bf0f7-0076-4d76-9561-c6e8b4ac2355 None admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:06.670913 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7bf0f7-0076-4d76-9561-c6e8b4ac2355 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:28:06.670913 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7bf0f7-0076-4d76-9561-c6e8b4ac2355 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:06.670913 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7bf0f7-0076-4d76-9561-c6e8b4ac2355 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:06.682872 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 628/1272] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Aug 24 05:28:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:28:06.688935 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f9677-d853-497f-b784-df6fa27428c3 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:06.786196 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:28:06.799699 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f9677-d853-497f-b784-df6fa27428c3 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:06.837191 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc658e2e-0294-4a07-b5c2-eac24bff2cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:06.837855 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc658e2e-0294-4a07-b5c2-eac24bff2cb4 None admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:06.838145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc658e2e-0294-4a07-b5c2-eac24bff2cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:06.838418 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc658e2e-0294-4a07-b5c2-eac24bff2cb4 None admin] PATH_INFO: `/v3/projects/59592706ae9442e08353158c625e9581/users/6778b7c5ad19484b877494f143050405/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:06.840434 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc658e2e-0294-4a07-b5c2-eac24bff2cb4 None admin] RBAC: Authorizing `identity:create_grant(project_id=59592706ae9442e08353158c625e9581, user_id=6778b7c5ad19484b877494f143050405, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:06.841817 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc658e2e-0294-4a07-b5c2-eac24bff2cb4 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:06.950571 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 645/1273] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Aug 24 05:28:06 2022] PUT /identity/v3/projects/59592706ae9442e08353158c625e9581/users/6778b7c5ad19484b877494f143050405/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:28:08.885727 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b05e4-e79e-4492-91b6-e839b0cdb2d5 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:08.886274 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b05e4-e79e-4492-91b6-e839b0cdb2d5 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:08.886274 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b05e4-e79e-4492-91b6-e839b0cdb2d5 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:08.887648 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 629/1274] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:28:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:28:08.893062 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3207dd3-889c-4c50-8316-4d8579b66668 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:08.894114 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3207dd3-889c-4c50-8316-4d8579b66668 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:08.894315 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3207dd3-889c-4c50-8316-4d8579b66668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:08.912678 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-e3207dd3-889c-4c50-8316-4d8579b66668 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:28:08.913883 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-e3207dd3-889c-4c50-8316-4d8579b66668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:28:08.981481 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 646/1275] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:28:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:28:09.254989 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b05e4-e79e-4492-91b6-e839b0cdb2d5 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:09.355063 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b05e4-e79e-4492-91b6-e839b0cdb2d5 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:09.373977 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:28:09.390417 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee14c2c-4acb-4b6c-b730-26739e5b4742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:09.391036 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee14c2c-4acb-4b6c-b730-26739e5b4742 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:09.391312 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee14c2c-4acb-4b6c-b730-26739e5b4742 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:09.391584 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee14c2c-4acb-4b6c-b730-26739e5b4742 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:09.401465 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee14c2c-4acb-4b6c-b730-26739e5b4742 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:28:09.405999 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-7ee14c2c-4acb-4b6c-b730-26739e5b4742 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:28:09.405999 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:28:09.405999 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:28:09.405999 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:28:09.405999 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:28:09.405999 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:28:09.405999 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:28:09.405999 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:28:09.405999 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:28:09.405999 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:28:09.405999 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:28:09.405999 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:28:09.405999 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:28:09.405999 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:28:09.405999 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:28:09.405999 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:28:09.406952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:28:09.406952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:28:09.406952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:28:09.406952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:28:09.406952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:28:09.406952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:28:09.406952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:28:09.406952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:28:09.406952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:28:09.406952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:28:09.406952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:28:09.406952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:28:09.406952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:28:09.406952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:28:09.406952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:28:09.406952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:28:09.408103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:28:09.408103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:28:09.408103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:28:09.408103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:28:09.408103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:28:09.408103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:28:09.408103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:28:09.408103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:28:09.408103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:28:09.408103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:28:09.408103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:28:09.408103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:28:09.408103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 630/1276] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Aug 24 05:28:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:28:09.414151 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3207dd3-889c-4c50-8316-4d8579b66668 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:09.496352 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:28:09.513727 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3207dd3-889c-4c50-8316-4d8579b66668 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:09.547719 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e0e2e-1382-4bb3-87a4-94ca10f6787f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:09.548368 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e0e2e-1382-4bb3-87a4-94ca10f6787f None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:09.548642 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e0e2e-1382-4bb3-87a4-94ca10f6787f None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:09.548919 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e0e2e-1382-4bb3-87a4-94ca10f6787f None admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:09.550219 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8e0e2e-1382-4bb3-87a4-94ca10f6787f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:28:09.550529 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8e0e2e-1382-4bb3-87a4-94ca10f6787f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:09.552057 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8e0e2e-1382-4bb3-87a4-94ca10f6787f None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:09.560139 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 647/1277] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Aug 24 05:28:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:28:09.566711 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee14c2c-4acb-4b6c-b730-26739e5b4742 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:09.669465 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee14c2c-4acb-4b6c-b730-26739e5b4742 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:09.703621 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f56ee-cee7-43c6-a37f-cff80f9e51bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:09.704242 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f56ee-cee7-43c6-a37f-cff80f9e51bf None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:09.704558 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f56ee-cee7-43c6-a37f-cff80f9e51bf None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:09.704837 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f56ee-cee7-43c6-a37f-cff80f9e51bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:09.715466 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f56ee-cee7-43c6-a37f-cff80f9e51bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:09.716953 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f56ee-cee7-43c6-a37f-cff80f9e51bf None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:09.731742 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-e03f56ee-cee7-43c6-a37f-cff80f9e51bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:28:09.731742 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:28:09.731742 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:28:09.731742 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:28:09.731742 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:28:09.731742 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:28:09.731742 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:28:09.731742 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:28:09.731742 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:28:09.731742 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:28:09.731742 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:28:09.731742 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:28:09.731742 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:28:09.731742 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:28:09.731742 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:28:09.731742 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:28:09.732982 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:28:09.732982 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:28:09.732982 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:28:09.732982 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:28:09.732982 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:28:09.732982 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:28:09.732982 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:28:09.732982 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:28:09.732982 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:28:09.732982 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:28:09.732982 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:28:09.732982 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:28:09.732982 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:28:09.732982 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:28:09.732982 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:28:09.732982 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:28:09.732982 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:28:09.734053 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:28:09.734053 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:28:09.734053 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:28:09.734053 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 631/1278] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:28:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:28:09.739037 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e0e2e-1382-4bb3-87a4-94ca10f6787f None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:09.842450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e0e2e-1382-4bb3-87a4-94ca10f6787f None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:09.875745 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eedd6ff-03ed-40d1-b8e0-b32720f7ef01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:09.876357 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eedd6ff-03ed-40d1-b8e0-b32720f7ef01 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:09.876673 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eedd6ff-03ed-40d1-b8e0-b32720f7ef01 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:09.876951 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eedd6ff-03ed-40d1-b8e0-b32720f7ef01 None admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:09.877594 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eedd6ff-03ed-40d1-b8e0-b32720f7ef01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:28:09.877927 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eedd6ff-03ed-40d1-b8e0-b32720f7ef01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:09.879303 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eedd6ff-03ed-40d1-b8e0-b32720f7ef01 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:09.890910 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 648/1279] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:28:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:28:09.897739 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f56ee-cee7-43c6-a37f-cff80f9e51bf None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:09.929640 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:28:09.998179 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f56ee-cee7-43c6-a37f-cff80f9e51bf None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:10.033407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367c4bd1-c0e2-49e0-bf60-e3d5b71b0dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:10.034623 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c4bd1-c0e2-49e0-bf60-e3d5b71b0dde None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:10.035185 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c4bd1-c0e2-49e0-bf60-e3d5b71b0dde None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:10.035537 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c4bd1-c0e2-49e0-bf60-e3d5b71b0dde None admin] PATH_INFO: `/v3/users/placement` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:10.042626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367c4bd1-c0e2-49e0-bf60-e3d5b71b0dde None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:10.044063 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367c4bd1-c0e2-49e0-bf60-e3d5b71b0dde None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:10.056210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-367c4bd1-c0e2-49e0-bf60-e3d5b71b0dde None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 05:28:10.056210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:28:10.056210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:28:10.056210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:28:10.056210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:28:10.056210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:28:10.056210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:28:10.056210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:28:10.056210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:28:10.056210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:28:10.056210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:28:10.056210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:28:10.056210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:28:10.056210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:28:10.056210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:28:10.056210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:28:10.059897 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:28:10.059897 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:28:10.059897 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:28:10.059897 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:28:10.059897 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:28:10.059897 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:28:10.059897 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:28:10.059897 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:28:10.059897 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:28:10.059897 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:28:10.059897 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:28:10.059897 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:28:10.059897 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:28:10.059897 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:28:10.059897 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:28:10.059897 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:28:10.059897 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:28:10.059897 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:28:10.061103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:28:10.061103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:28:10.061103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:28:10.061103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:28:10.061103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:28:10.061103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:28:10.061103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:28:10.061103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:28:10.061103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 05:28:10.061103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:28:10.061103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 632/1280] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:28:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:28:10.063429 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eedd6ff-03ed-40d1-b8e0-b32720f7ef01 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:10.067292 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:28:10.165427 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eedd6ff-03ed-40d1-b8e0-b32720f7ef01 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:10.200911 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41739e85-c175-4432-b5fc-57d260e65952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:10.201515 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41739e85-c175-4432-b5fc-57d260e65952 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:10.201796 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41739e85-c175-4432-b5fc-57d260e65952 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:10.202068 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41739e85-c175-4432-b5fc-57d260e65952 None admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:10.202781 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41739e85-c175-4432-b5fc-57d260e65952 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:28:10.203085 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41739e85-c175-4432-b5fc-57d260e65952 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:10.204519 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41739e85-c175-4432-b5fc-57d260e65952 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:10.221812 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 649/1281] 10.222.0.15 () {56 vars in 1143 bytes} [Wed Aug 24 05:28:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:28:10.227780 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367c4bd1-c0e2-49e0-bf60-e3d5b71b0dde None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:10.320671 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:28:10.330197 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367c4bd1-c0e2-49e0-bf60-e3d5b71b0dde None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:10.364054 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0603841b-4a84-45b1-a5a0-3acb03c1b75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:10.364688 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0603841b-4a84-45b1-a5a0-3acb03c1b75e None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:10.364964 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0603841b-4a84-45b1-a5a0-3acb03c1b75e None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:10.365236 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0603841b-4a84-45b1-a5a0-3acb03c1b75e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:10.430575 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0603841b-4a84-45b1-a5a0-3acb03c1b75e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:10.431949 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0603841b-4a84-45b1-a5a0-3acb03c1b75e None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:10.528576 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:28:10.539655 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 633/1282] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Aug 24 05:28:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:28:10.545781 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41739e85-c175-4432-b5fc-57d260e65952 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:10.606332 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:28:10.652034 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41739e85-c175-4432-b5fc-57d260e65952 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:10.686144 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd1c251-93a1-41b4-ae74-ae6f681752ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:10.686706 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd1c251-93a1-41b4-ae74-ae6f681752ec None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:10.686988 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd1c251-93a1-41b4-ae74-ae6f681752ec None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:10.687265 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd1c251-93a1-41b4-ae74-ae6f681752ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:10.698605 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd1c251-93a1-41b4-ae74-ae6f681752ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:10.700043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd1c251-93a1-41b4-ae74-ae6f681752ec None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:10.714343 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-8fd1c251-93a1-41b4-ae74-ae6f681752ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:28:10.714343 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:28:10.714343 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:28:10.714343 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:28:10.714343 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:28:10.714343 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:28:10.714343 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:28:10.714343 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:28:10.714343 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:28:10.714343 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:28:10.714343 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:28:10.714343 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:28:10.714343 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:28:10.714343 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:28:10.714343 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:28:10.714343 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:28:10.721803 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:28:10.721803 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:28:10.721803 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:28:10.721803 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:28:10.721803 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:28:10.721803 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:28:10.721803 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:28:10.721803 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:28:10.721803 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:28:10.721803 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:28:10.721803 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:28:10.721803 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:28:10.721803 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:28:10.721803 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:28:10.721803 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:28:10.721803 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:28:10.721803 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:28:10.723082 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:28:10.723082 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:28:10.723082 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:28:10.723082 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 650/1283] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:28:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:28:10.723451 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0603841b-4a84-45b1-a5a0-3acb03c1b75e None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:10.822903 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0603841b-4a84-45b1-a5a0-3acb03c1b75e None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:10.855872 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b091901-a9df-4251-be4d-d85f16d018fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:10.856497 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b091901-a9df-4251-be4d-d85f16d018fd None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:10.856775 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b091901-a9df-4251-be4d-d85f16d018fd None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:10.857049 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b091901-a9df-4251-be4d-d85f16d018fd None admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:10.857723 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b091901-a9df-4251-be4d-d85f16d018fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:28:10.858026 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b091901-a9df-4251-be4d-d85f16d018fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:10.859375 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b091901-a9df-4251-be4d-d85f16d018fd None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:10.871560 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 634/1284] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Aug 24 05:28:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:28:10.877300 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd1c251-93a1-41b4-ae74-ae6f681752ec None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:10.980025 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd1c251-93a1-41b4-ae74-ae6f681752ec None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:10.994315 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:28:11.013985 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd0dff-f0ed-4eaa-90c2-637321115a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:11.014646 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd0dff-f0ed-4eaa-90c2-637321115a19 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:11.014927 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd0dff-f0ed-4eaa-90c2-637321115a19 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:11.015199 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd0dff-f0ed-4eaa-90c2-637321115a19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:11.021772 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cd0dff-f0ed-4eaa-90c2-637321115a19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:11.023276 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cd0dff-f0ed-4eaa-90c2-637321115a19 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:11.034879 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application [None req-e1cd0dff-f0ed-4eaa-90c2-637321115a19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:28:11.034879 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:28:11.034879 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:28:11.034879 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:28:11.034879 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:28:11.034879 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:28:11.034879 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:28:11.034879 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:28:11.034879 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:28:11.034879 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:28:11.034879 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:28:11.034879 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:28:11.034879 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:28:11.034879 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:28:11.034879 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:28:11.035973 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:28:11.035973 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:28:11.035973 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:28:11.035973 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:28:11.035973 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:28:11.035973 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:28:11.035973 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:28:11.035973 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:28:11.035973 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application with Lock( Aug 24 05:28:11.035973 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:28:11.035973 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:28:11.035973 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:28:11.035973 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:28:11.035973 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:28:11.035973 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:28:11.035973 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:28:11.035973 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:28:11.037111 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:28:11.037111 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:28:11.037111 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:28:11.037111 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:28:11.037111 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:28:11.037111 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:28:11.037111 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:28:11.037111 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: ERROR keystone.server.flask.application Aug 24 05:28:11.037111 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 651/1285] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Aug 24 05:28:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:28:11.042142 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b091901-a9df-4251-be4d-d85f16d018fd None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:11.090919 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:28:11.139338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b091901-a9df-4251-be4d-d85f16d018fd None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:11.172834 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c8cf8-e806-4b53-a3bc-6d6000ddbe63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:11.173466 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c8cf8-e806-4b53-a3bc-6d6000ddbe63 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:11.173743 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c8cf8-e806-4b53-a3bc-6d6000ddbe63 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:11.174012 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c8cf8-e806-4b53-a3bc-6d6000ddbe63 None admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:11.175470 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8c8cf8-e806-4b53-a3bc-6d6000ddbe63 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:28:11.176211 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8c8cf8-e806-4b53-a3bc-6d6000ddbe63 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:11.177752 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8c8cf8-e806-4b53-a3bc-6d6000ddbe63 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:11.191808 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 635/1286] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Aug 24 05:28:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:28:11.198081 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd0dff-f0ed-4eaa-90c2-637321115a19 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:11.298548 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd0dff-f0ed-4eaa-90c2-637321115a19 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:11.331832 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d25ac8-1fc1-431d-be57-6e8529809943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:11.332449 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d25ac8-1fc1-431d-be57-6e8529809943 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:11.332736 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d25ac8-1fc1-431d-be57-6e8529809943 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:11.333012 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d25ac8-1fc1-431d-be57-6e8529809943 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:11.333735 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d25ac8-1fc1-431d-be57-6e8529809943 None admin] RBAC: Adding query filter params (role.id=e013ad4d436f4cb0a9bf7debb7d13dc7, scope.project.id=59592706ae9442e08353158c625e9581, user.id=6778b7c5ad19484b877494f143050405) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:28:11.334046 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d25ac8-1fc1-431d-be57-6e8529809943 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:11.335480 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d25ac8-1fc1-431d-be57-6e8529809943 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:11.341322 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 652/1287] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Aug 24 05:28:11 2022] GET /identity/v3/role_assignments?user.id=6778b7c5ad19484b877494f143050405&scope.project.id=59592706ae9442e08353158c625e9581&role.id=e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:28:13.266520 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257b6b1-b31e-4848-a55e-914f42679720 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:13.267038 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257b6b1-b31e-4848-a55e-914f42679720 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:13.267038 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257b6b1-b31e-4848-a55e-914f42679720 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:13.269051 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 636/1288] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:28:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:28:13.274206 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362d548-3b8c-4bd9-98d2-ef94981fdda5 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:13.274614 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362d548-3b8c-4bd9-98d2-ef94981fdda5 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:13.274771 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362d548-3b8c-4bd9-98d2-ef94981fdda5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:13.293558 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-2362d548-3b8c-4bd9-98d2-ef94981fdda5 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:28:13.296119 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-2362d548-3b8c-4bd9-98d2-ef94981fdda5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:28:13.324821 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:28:13.361370 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 653/1289] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:28:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:28:13.626593 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c257b6b1-b31e-4848-a55e-914f42679720 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:13.732531 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c257b6b1-b31e-4848-a55e-914f42679720 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:13.744102 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:28:13.767653 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c3c9f-82e2-4e31-aa5c-19c0f9735e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:13.768495 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c3c9f-82e2-4e31-aa5c-19c0f9735e55 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:13.768976 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c3c9f-82e2-4e31-aa5c-19c0f9735e55 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:13.769501 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c3c9f-82e2-4e31-aa5c-19c0f9735e55 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:13.770255 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28c3c9f-82e2-4e31-aa5c-19c0f9735e55 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:13.771623 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28c3c9f-82e2-4e31-aa5c-19c0f9735e55 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:13.780119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-e28c3c9f-82e2-4e31-aa5c-19c0f9735e55 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 05:28:13.780119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:28:13.780119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:28:13.780119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:28:13.780119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:28:13.780119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:28:13.780119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:28:13.780119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:28:13.780119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 05:28:13.780119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:28:13.780119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:28:13.780119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:28:13.780119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:28:13.780119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:28:13.780119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:28:13.780119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:28:13.780119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:28:13.781413 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:28:13.781413 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:28:13.781413 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:28:13.781413 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:28:13.781413 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:28:13.781413 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:28:13.781413 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:28:13.781413 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:28:13.781413 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:28:13.781413 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:28:13.781413 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:28:13.781413 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:28:13.781413 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:28:13.781413 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:28:13.781413 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:28:13.781413 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:28:13.783032 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:28:13.783032 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:28:13.783032 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:28:13.783032 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:28:13.783032 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:28:13.783032 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:28:13.783032 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:28:13.783032 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:28:13.783032 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:28:13.783032 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:28:13.783032 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:28:13.783032 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:28:13.783032 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 05:28:13.783032 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:28:13.783032 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 637/1290] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Aug 24 05:28:13 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:28:13.788062 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362d548-3b8c-4bd9-98d2-ef94981fdda5 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:13.887756 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362d548-3b8c-4bd9-98d2-ef94981fdda5 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:13.921463 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d3db8-f90c-4093-bd45-0c9220d4a218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:13.922066 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d3db8-f90c-4093-bd45-0c9220d4a218 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:13.922355 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d3db8-f90c-4093-bd45-0c9220d4a218 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:13.922628 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d3db8-f90c-4093-bd45-0c9220d4a218 None admin] PATH_INFO: `/v3/services` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:13.923257 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d3db8-f90c-4093-bd45-0c9220d4a218 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:28:13.923565 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d3db8-f90c-4093-bd45-0c9220d4a218 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:13.924993 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d3db8-f90c-4093-bd45-0c9220d4a218 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:13.929853 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 654/1291] 10.222.0.15 () {56 vars in 1116 bytes} [Wed Aug 24 05:28:13 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:28:13.935984 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c3c9f-82e2-4e31-aa5c-19c0f9735e55 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:14.035181 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c3c9f-82e2-4e31-aa5c-19c0f9735e55 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:14.068133 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbbdcb3-e3c4-403c-974c-25045ddadd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:14.068839 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbbdcb3-e3c4-403c-974c-25045ddadd0c None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:14.069118 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbbdcb3-e3c4-403c-974c-25045ddadd0c None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:14.069295 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbbdcb3-e3c4-403c-974c-25045ddadd0c None admin] PATH_INFO: `/v3/services` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:14.069910 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbbdcb3-e3c4-403c-974c-25045ddadd0c None admin] RBAC: Adding query filter params (type=placement) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:28:14.070209 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbbdcb3-e3c4-403c-974c-25045ddadd0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:14.071563 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbbdcb3-e3c4-403c-974c-25045ddadd0c None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:14.076038 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 638/1292] 10.222.0.15 () {56 vars in 1116 bytes} [Wed Aug 24 05:28:13 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:28:15.985143 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efba0d-c701-46cc-8712-1b07d10623b6 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:15.985710 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efba0d-c701-46cc-8712-1b07d10623b6 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:15.985710 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efba0d-c701-46cc-8712-1b07d10623b6 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:15.987622 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 655/1293] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:28:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:28:15.992775 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acac142-83c9-4824-a0ec-36b59c5c7d0f None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:15.993177 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acac142-83c9-4824-a0ec-36b59c5c7d0f None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:15.993330 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acac142-83c9-4824-a0ec-36b59c5c7d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:16.011678 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-9acac142-83c9-4824-a0ec-36b59c5c7d0f None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:28:16.012893 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-9acac142-83c9-4824-a0ec-36b59c5c7d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:28:16.042067 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:28:16.078653 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 639/1294] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:28:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:28:16.343207 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79efba0d-c701-46cc-8712-1b07d10623b6 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:16.374345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:28:16.442621 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79efba0d-c701-46cc-8712-1b07d10623b6 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:16.476954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f127fd96-32e7-4a33-a665-7b6e6b015df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:16.477565 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127fd96-32e7-4a33-a665-7b6e6b015df0 None admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:16.477844 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127fd96-32e7-4a33-a665-7b6e6b015df0 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:16.478115 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127fd96-32e7-4a33-a665-7b6e6b015df0 None admin] PATH_INFO: `/v3/services` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:16.478762 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f127fd96-32e7-4a33-a665-7b6e6b015df0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:16.480152 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f127fd96-32e7-4a33-a665-7b6e6b015df0 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:16.586234 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 656/1295] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Aug 24 05:28:16 2022] POST /identity/v3/services => generated 246 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:28:18.592746 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b788d1f-a029-4409-ba62-9667352a36ed None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:18.593301 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b788d1f-a029-4409-ba62-9667352a36ed None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:18.593301 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b788d1f-a029-4409-ba62-9667352a36ed None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:18.594570 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 640/1296] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:28:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:28:18.600956 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884c77c-c905-49d6-a3f2-5d3ccf09986c None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:18.601428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884c77c-c905-49d6-a3f2-5d3ccf09986c None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:18.601573 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884c77c-c905-49d6-a3f2-5d3ccf09986c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:18.622358 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-2884c77c-c905-49d6-a3f2-5d3ccf09986c None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:28:18.623678 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-2884c77c-c905-49d6-a3f2-5d3ccf09986c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:28:18.699255 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 657/1297] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:28:18 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:28:18.989110 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b788d1f-a029-4409-ba62-9667352a36ed None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:19.095515 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b788d1f-a029-4409-ba62-9667352a36ed None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:19.131522 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dca9c72-a6cd-4beb-adf2-285f81fbcb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:19.132150 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca9c72-a6cd-4beb-adf2-285f81fbcb3f None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:19.132456 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca9c72-a6cd-4beb-adf2-285f81fbcb3f None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:19.132737 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca9c72-a6cd-4beb-adf2-285f81fbcb3f None admin] PATH_INFO: `/v3/services/placement` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:19.133321 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dca9c72-a6cd-4beb-adf2-285f81fbcb3f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:19.134707 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dca9c72-a6cd-4beb-adf2-285f81fbcb3f None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:19.143768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-2dca9c72-a6cd-4beb-adf2-285f81fbcb3f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 05:28:19.143768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:28:19.143768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:28:19.143768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:28:19.143768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:28:19.143768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:28:19.143768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:28:19.143768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:28:19.143768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 05:28:19.143768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:28:19.143768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:28:19.143768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:28:19.143768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:28:19.143768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:28:19.143768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:28:19.143768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:28:19.143768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:28:19.152196 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:28:19.152196 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:28:19.152196 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:28:19.152196 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:28:19.152196 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:28:19.152196 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:28:19.152196 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:28:19.152196 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:28:19.152196 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:28:19.152196 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:28:19.152196 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:28:19.152196 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:28:19.152196 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:28:19.152196 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:28:19.152196 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:28:19.152196 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:28:19.153343 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:28:19.153343 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:28:19.153343 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:28:19.153343 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:28:19.153343 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:28:19.153343 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:28:19.153343 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:28:19.153343 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:28:19.153343 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:28:19.153343 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:28:19.153343 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:28:19.153343 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:28:19.153343 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 05:28:19.153343 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:28:19.153343 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 641/1298] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Aug 24 05:28:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:28:19.154453 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2884c77c-c905-49d6-a3f2-5d3ccf09986c None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:19.171084 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:28:19.251881 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2884c77c-c905-49d6-a3f2-5d3ccf09986c None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:19.287304 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58c4e1b-6e06-47fe-8d62-46b0fa9fcee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:19.287905 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58c4e1b-6e06-47fe-8d62-46b0fa9fcee4 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:19.288183 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58c4e1b-6e06-47fe-8d62-46b0fa9fcee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:19.288486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58c4e1b-6e06-47fe-8d62-46b0fa9fcee4 None admin] PATH_INFO: `/v3/services` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:19.289116 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58c4e1b-6e06-47fe-8d62-46b0fa9fcee4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:28:19.289420 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58c4e1b-6e06-47fe-8d62-46b0fa9fcee4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:19.290811 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58c4e1b-6e06-47fe-8d62-46b0fa9fcee4 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:19.295746 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 658/1299] 10.222.0.15 () {56 vars in 1116 bytes} [Wed Aug 24 05:28:19 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:28:19.301884 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dca9c72-a6cd-4beb-adf2-285f81fbcb3f None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:19.336058 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:28:19.409156 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dca9c72-a6cd-4beb-adf2-285f81fbcb3f None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:19.442629 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5624eb2a-8171-44c7-87b0-bfa7519a972d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:19.443223 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624eb2a-8171-44c7-87b0-bfa7519a972d None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:19.443500 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624eb2a-8171-44c7-87b0-bfa7519a972d None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:19.443775 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624eb2a-8171-44c7-87b0-bfa7519a972d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:19.444494 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5624eb2a-8171-44c7-87b0-bfa7519a972d None admin] RBAC: Adding query filter params (interface=public, service_id=1e10c3f53a674782ba7e2fa6b101109a, region_id=RegionOne) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:28:19.444800 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5624eb2a-8171-44c7-87b0-bfa7519a972d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:19.446165 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5624eb2a-8171-44c7-87b0-bfa7519a972d None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:19.451140 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 642/1300] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Aug 24 05:28:19 2022] GET /identity/v3/endpoints?service_id=1e10c3f53a674782ba7e2fa6b101109a&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:28:19.457589 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58c4e1b-6e06-47fe-8d62-46b0fa9fcee4 None admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:19.541108 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:28:19.559484 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58c4e1b-6e06-47fe-8d62-46b0fa9fcee4 None admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:19.594456 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e03acaa-9568-4106-a9b7-d3cd3ee483a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:19.595119 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e03acaa-9568-4106-a9b7-d3cd3ee483a4 None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:19.595440 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e03acaa-9568-4106-a9b7-d3cd3ee483a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:19.595731 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e03acaa-9568-4106-a9b7-d3cd3ee483a4 None admin] PATH_INFO: `/v3/services` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:19.597013 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e03acaa-9568-4106-a9b7-d3cd3ee483a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:19.598458 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e03acaa-9568-4106-a9b7-d3cd3ee483a4 None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:19.606119 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 659/1301] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Aug 24 05:28:19 2022] GET /identity/v3/services => generated 2191 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:28:21.578568 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea187a-64fc-451a-adbf-e637d355194e None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:21.579076 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea187a-64fc-451a-adbf-e637d355194e None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:21.579076 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea187a-64fc-451a-adbf-e637d355194e None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:21.580387 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 643/1302] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:28:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:28:21.586240 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f914ef-ac84-4241-951e-9bedf93a63ab None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:21.586697 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f914ef-ac84-4241-951e-9bedf93a63ab None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:21.586787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f914ef-ac84-4241-951e-9bedf93a63ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:21.605021 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-69f914ef-ac84-4241-951e-9bedf93a63ab None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:28:21.606212 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-69f914ef-ac84-4241-951e-9bedf93a63ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:28:21.674122 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 660/1303] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:28:21 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:28:21.946257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea187a-64fc-451a-adbf-e637d355194e None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:22.041235 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:28:22.047727 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea187a-64fc-451a-adbf-e637d355194e None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:22.081963 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd7aed-0c23-4174-9839-505afaa1d7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:22.082575 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd7aed-0c23-4174-9839-505afaa1d7b3 None admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:22.082852 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd7aed-0c23-4174-9839-505afaa1d7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:22.083120 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd7aed-0c23-4174-9839-505afaa1d7b3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:22.083691 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffd7aed-0c23-4174-9839-505afaa1d7b3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:22.085060 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffd7aed-0c23-4174-9839-505afaa1d7b3 None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:22.093524 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application [None req-5ffd7aed-0c23-4174-9839-505afaa1d7b3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 05:28:22.093524 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:28:22.093524 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:28:22.093524 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:28:22.093524 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:28:22.093524 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:28:22.093524 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:28:22.093524 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:28:22.093524 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 05:28:22.093524 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:28:22.093524 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:28:22.093524 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:28:22.093524 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:28:22.093524 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:28:22.093524 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:28:22.093524 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:28:22.093524 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:28:22.094807 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:28:22.094807 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:28:22.094807 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:28:22.094807 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:28:22.094807 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:28:22.094807 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:28:22.094807 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:28:22.094807 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:28:22.094807 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:28:22.094807 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:28:22.094807 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:28:22.094807 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:28:22.094807 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:28:22.094807 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:28:22.094807 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:28:22.094807 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:28:22.095843 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:28:22.095843 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application with Lock( Aug 24 05:28:22.095843 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:28:22.095843 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:28:22.095843 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:28:22.095843 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:28:22.095843 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:28:22.095843 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:28:22.095843 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:28:22.095843 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:28:22.095843 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:28:22.095843 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:28:22.095843 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 05:28:22.095843 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: ERROR keystone.server.flask.application Aug 24 05:28:22.095843 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 644/1304] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Aug 24 05:28:21 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:28:22.101548 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f914ef-ac84-4241-951e-9bedf93a63ab None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:22.184157 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:28:22.211497 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f914ef-ac84-4241-951e-9bedf93a63ab None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:22.243101 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f605b-3908-4df5-b53c-0d92810eccfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:22.243702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f605b-3908-4df5-b53c-0d92810eccfd None admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:22.243985 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f605b-3908-4df5-b53c-0d92810eccfd None admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:22.244263 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f605b-3908-4df5-b53c-0d92810eccfd None admin] PATH_INFO: `/v3/services` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:22.244929 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673f605b-3908-4df5-b53c-0d92810eccfd None admin] RBAC: Adding query filter params (name=placement) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:28:22.245232 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673f605b-3908-4df5-b53c-0d92810eccfd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:22.246591 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673f605b-3908-4df5-b53c-0d92810eccfd None admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:22.251552 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 661/1305] 10.222.0.15 () {56 vars in 1116 bytes} [Wed Aug 24 05:28:22 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:28:22.258206 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd7aed-0c23-4174-9839-505afaa1d7b3 None admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:28:22.361387 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd7aed-0c23-4174-9839-505afaa1d7b3 None admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:28:22.395243 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b012ecd8-3d65-4f69-9606-1176c75a9efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:28:22.395843 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012ecd8-3d65-4f69-9606-1176c75a9efa None admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:28:22.396125 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012ecd8-3d65-4f69-9606-1176c75a9efa None admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:28:22.396426 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012ecd8-3d65-4f69-9606-1176c75a9efa None admin] PATH_INFO: `/v3/endpoints` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:28:22.397095 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b012ecd8-3d65-4f69-9606-1176c75a9efa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:28:22.398461 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b012ecd8-3d65-4f69-9606-1176c75a9efa None admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:28:22.533813 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 645/1306] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:28:22 2022] POST /identity/v3/endpoints => generated 331 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:28:32.292563 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=30 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:28:32.528442 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:35:50.777354 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea5977-b18b-4290-80fb-b99788bc222f None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:50.778409 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea5977-b18b-4290-80fb-b99788bc222f None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:50.778931 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea5977-b18b-4290-80fb-b99788bc222f None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:50.780956 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 662/1307] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:35:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:35:50.788148 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92fe604-d799-4a4d-a3f3-62a50e3fe759 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:50.788706 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92fe604-d799-4a4d-a3f3-62a50e3fe759 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:50.788791 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92fe604-d799-4a4d-a3f3-62a50e3fe759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:50.872416 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-d92fe604-d799-4a4d-a3f3-62a50e3fe759 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:35:50.873648 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-d92fe604-d799-4a4d-a3f3-62a50e3fe759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:35:50.986225 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 646/1308] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:35:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:35:51.312883 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96f483-a1cf-4031-aef6-1970ae33198f None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:51.313267 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96f483-a1cf-4031-aef6-1970ae33198f None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:51.313473 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96f483-a1cf-4031-aef6-1970ae33198f None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:51.319529 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 663/1309] 10.222.0.15 () {54 vars in 965 bytes} [Wed Aug 24 05:35:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:35:51.326129 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc3e6e-3505-46e9-be58-ae40244f6df7 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:51.326818 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc3e6e-3505-46e9-be58-ae40244f6df7 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:51.327180 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc3e6e-3505-46e9-be58-ae40244f6df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:51.413571 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-9edc3e6e-3505-46e9-be58-ae40244f6df7 None None] MFA Rules not processed for user `b1ecd9b302b64ecf81d7e3d3356ddf7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:35:51.414731 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-9edc3e6e-3505-46e9-be58-ae40244f6df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:35:51.448420 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:35:51.511351 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 647/1310] 10.222.0.15 () {58 vars in 1062 bytes} [Wed Aug 24 05:35:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:35:51.520940 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e96f483-a1cf-4031-aef6-1970ae33198f None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:35:51.522683 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-9e96f483-a1cf-4031-aef6-1970ae33198f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:35:51.652932 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e96f483-a1cf-4031-aef6-1970ae33198f None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:35:51.742729 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a841e5-1265-4e0e-9544-2942e5b30d19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:35:51.744197 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a841e5-1265-4e0e-9544-2942e5b30d19 service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:51.744873 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a841e5-1265-4e0e-9544-2942e5b30d19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:51.745395 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a841e5-1265-4e0e-9544-2942e5b30d19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:51.747427 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-b4a841e5-1265-4e0e-9544-2942e5b30d19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:35:51.811939 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:35:51.869996 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a841e5-1265-4e0e-9544-2942e5b30d19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:35:51.871439 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a841e5-1265-4e0e-9544-2942e5b30d19 service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:35:51.891975 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 664/1311] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:35:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:35:54.206937 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1d49b-2133-4f22-9b29-c423e3b33712 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:54.208178 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1d49b-2133-4f22-9b29-c423e3b33712 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:54.208803 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1d49b-2133-4f22-9b29-c423e3b33712 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:54.210403 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 648/1312] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:35:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:35:54.216315 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0563ac0-f75b-4436-9cbd-3f0fda34d1da None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:54.216866 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0563ac0-f75b-4436-9cbd-3f0fda34d1da None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:54.217057 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0563ac0-f75b-4436-9cbd-3f0fda34d1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:54.305899 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-d0563ac0-f75b-4436-9cbd-3f0fda34d1da None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:35:54.307142 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-d0563ac0-f75b-4436-9cbd-3f0fda34d1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:35:54.407176 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 665/1313] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:35:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:35:54.696276 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d1d49b-2133-4f22-9b29-c423e3b33712 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:35:54.697837 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-c0d1d49b-2133-4f22-9b29-c423e3b33712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:35:54.802539 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d1d49b-2133-4f22-9b29-c423e3b33712 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:35:54.872069 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:35:54.874580 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6247e4eb-2d55-417a-bdfa-c6c9cdd9ee0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:35:54.875181 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247e4eb-2d55-417a-bdfa-c6c9cdd9ee0f service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:54.875458 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247e4eb-2d55-417a-bdfa-c6c9cdd9ee0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:54.875732 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247e4eb-2d55-417a-bdfa-c6c9cdd9ee0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:54.877375 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-6247e4eb-2d55-417a-bdfa-c6c9cdd9ee0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:35:54.985156 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6247e4eb-2d55-417a-bdfa-c6c9cdd9ee0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:35:54.986594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6247e4eb-2d55-417a-bdfa-c6c9cdd9ee0f service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:35:55.006233 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 649/1314] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:35:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:35:57.494108 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d581d-5c92-4047-b502-d3333e5cbe28 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:57.494578 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d581d-5c92-4047-b502-d3333e5cbe28 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:57.495238 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d581d-5c92-4047-b502-d3333e5cbe28 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:57.495997 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 666/1315] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:35:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:35:57.501361 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4d03c-fdba-4c56-9012-29c12105541f None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:57.501788 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4d03c-fdba-4c56-9012-29c12105541f None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:57.502049 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4d03c-fdba-4c56-9012-29c12105541f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:57.567787 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-03e4d03c-fdba-4c56-9012-29c12105541f None None] MFA Rules not processed for user `ab58de7f803c4cd29f61931966b168d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:35:57.568993 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-03e4d03c-fdba-4c56-9012-29c12105541f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:35:57.620982 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:35:57.665516 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 650/1316] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:35:57 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:35:57.968305 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d581d-5c92-4047-b502-d3333e5cbe28 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:35:57.969879 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-f04d581d-5c92-4047-b502-d3333e5cbe28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:35:57.980084 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:35:58.076642 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d581d-5c92-4047-b502-d3333e5cbe28 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:35:58.154985 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64146069-b610-48ba-8f9e-1b3b20a8610c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:35:58.155602 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64146069-b610-48ba-8f9e-1b3b20a8610c service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:58.155883 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64146069-b610-48ba-8f9e-1b3b20a8610c service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:58.156152 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64146069-b610-48ba-8f9e-1b3b20a8610c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:58.157813 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-64146069-b610-48ba-8f9e-1b3b20a8610c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:35:58.223116 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:35:58.269297 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64146069-b610-48ba-8f9e-1b3b20a8610c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:35:58.271448 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64146069-b610-48ba-8f9e-1b3b20a8610c service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:35:58.291340 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 667/1317] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:35:57 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:36:00.936344 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae3da90-373c-4321-b405-8fd0853dfac4 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:00.936914 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae3da90-373c-4321-b405-8fd0853dfac4 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:00.936914 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae3da90-373c-4321-b405-8fd0853dfac4 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:00.938265 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 651/1318] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:36:00.943629 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf35a1c-b65f-43bb-b51e-c373e037f4b7 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:00.944084 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf35a1c-b65f-43bb-b51e-c373e037f4b7 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:00.944182 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf35a1c-b65f-43bb-b51e-c373e037f4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:01.010466 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-acf35a1c-b65f-43bb-b51e-c373e037f4b7 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:36:01.011676 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-acf35a1c-b65f-43bb-b51e-c373e037f4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:01.109469 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 668/1319] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:36:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:36:01.407520 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bdcbfe-d1b1-4078-bc20-1f32392de413 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:01.407813 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bdcbfe-d1b1-4078-bc20-1f32392de413 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:01.408140 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bdcbfe-d1b1-4078-bc20-1f32392de413 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:01.409530 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 652/1320] 10.222.0.15 () {54 vars in 965 bytes} [Wed Aug 24 05:36:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:36:01.415937 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307b572-93f3-49f1-8075-4c79ce31d55a None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:01.416213 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307b572-93f3-49f1-8075-4c79ce31d55a None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:01.416535 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307b572-93f3-49f1-8075-4c79ce31d55a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:01.439206 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:01.494849 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-0307b572-93f3-49f1-8075-4c79ce31d55a None None] MFA Rules not processed for user `b1ecd9b302b64ecf81d7e3d3356ddf7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:36:01.496054 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-0307b572-93f3-49f1-8075-4c79ce31d55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:01.572583 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 669/1321] 10.222.0.15 () {58 vars in 1062 bytes} [Wed Aug 24 05:36:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:36:01.581958 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bdcbfe-d1b1-4078-bc20-1f32392de413 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:01.583842 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-f8bdcbfe-d1b1-4078-bc20-1f32392de413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:01.690335 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bdcbfe-d1b1-4078-bc20-1f32392de413 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:01.762991 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4679575b-1a55-44d2-a9ea-8dfa726bbb0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:01.763593 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4679575b-1a55-44d2-a9ea-8dfa726bbb0b service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:01.763875 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4679575b-1a55-44d2-a9ea-8dfa726bbb0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:01.764148 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4679575b-1a55-44d2-a9ea-8dfa726bbb0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:01.765776 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-4679575b-1a55-44d2-a9ea-8dfa726bbb0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:01.773848 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:01.877836 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4679575b-1a55-44d2-a9ea-8dfa726bbb0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:01.879858 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4679575b-1a55-44d2-a9ea-8dfa726bbb0b service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:01.898390 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 653/1322] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:36:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:36:05.265416 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1dc3ed-967a-4a24-b4ce-d2e8f5f57b89 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:05.265968 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1dc3ed-967a-4a24-b4ce-d2e8f5f57b89 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:05.265968 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1dc3ed-967a-4a24-b4ce-d2e8f5f57b89 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:05.267318 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 670/1323] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:36:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:36:05.273358 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12efa3b9-7cbb-4b9c-8e98-a806a5c49e2a None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:05.273811 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12efa3b9-7cbb-4b9c-8e98-a806a5c49e2a None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:05.273892 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12efa3b9-7cbb-4b9c-8e98-a806a5c49e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:05.341005 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-12efa3b9-7cbb-4b9c-8e98-a806a5c49e2a None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:36:05.342200 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-12efa3b9-7cbb-4b9c-8e98-a806a5c49e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:05.409044 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:05.448288 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 654/1324] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:36:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:36:05.755444 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1dc3ed-967a-4a24-b4ce-d2e8f5f57b89 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:05.757265 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-2d1dc3ed-967a-4a24-b4ce-d2e8f5f57b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:05.868417 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1dc3ed-967a-4a24-b4ce-d2e8f5f57b89 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:05.881688 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:05.945922 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b110947c-1768-4b1f-91c0-74101ea4119f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:05.946529 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b110947c-1768-4b1f-91c0-74101ea4119f service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:05.946813 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b110947c-1768-4b1f-91c0-74101ea4119f service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:05.947085 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b110947c-1768-4b1f-91c0-74101ea4119f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:05.948814 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-b110947c-1768-4b1f-91c0-74101ea4119f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:06.057191 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b110947c-1768-4b1f-91c0-74101ea4119f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:06.058534 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b110947c-1768-4b1f-91c0-74101ea4119f service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:06.077509 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 671/1325] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:36:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:36:08.045181 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68483ff4-7cc8-4a8d-9430-cd9f0ac34a7c None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:08.045699 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68483ff4-7cc8-4a8d-9430-cd9f0ac34a7c None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:08.045781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68483ff4-7cc8-4a8d-9430-cd9f0ac34a7c None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:08.047204 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 655/1326] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:36:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:36:08.053169 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6138d2b-627a-4163-8f64-cd14757bb3b2 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:08.053601 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6138d2b-627a-4163-8f64-cd14757bb3b2 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:08.053705 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6138d2b-627a-4163-8f64-cd14757bb3b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:08.104213 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:08.128278 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-d6138d2b-627a-4163-8f64-cd14757bb3b2 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:36:08.130206 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-d6138d2b-627a-4163-8f64-cd14757bb3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:08.243686 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 672/1327] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:36:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:36:08.524585 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68483ff4-7cc8-4a8d-9430-cd9f0ac34a7c None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:08.526126 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-68483ff4-7cc8-4a8d-9430-cd9f0ac34a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:08.642855 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68483ff4-7cc8-4a8d-9430-cd9f0ac34a7c None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:08.719562 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fdb27f-bbab-4e6f-971b-1f58bf9a6077 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:08.720156 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fdb27f-bbab-4e6f-971b-1f58bf9a6077 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:08.720458 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fdb27f-bbab-4e6f-971b-1f58bf9a6077 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:08.720731 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fdb27f-bbab-4e6f-971b-1f58bf9a6077 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:08.723150 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-e4fdb27f-bbab-4e6f-971b-1f58bf9a6077 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:08.741713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:08.840381 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fdb27f-bbab-4e6f-971b-1f58bf9a6077 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:08.841857 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fdb27f-bbab-4e6f-971b-1f58bf9a6077 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:08.861277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 656/1328] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:36:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:36:10.870476 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f198b56-3140-4bea-9abd-c4a6f7a9e57a None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:10.871083 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f198b56-3140-4bea-9abd-c4a6f7a9e57a None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:10.871083 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f198b56-3140-4bea-9abd-c4a6f7a9e57a None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:10.872618 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 673/1329] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:36:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:36:10.877943 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6909911-2252-4ea5-bc59-af892a635ecc None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:10.878975 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6909911-2252-4ea5-bc59-af892a635ecc None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:10.879191 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6909911-2252-4ea5-bc59-af892a635ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:10.946541 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-a6909911-2252-4ea5-bc59-af892a635ecc None None] MFA Rules not processed for user `ab58de7f803c4cd29f61931966b168d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:36:10.947754 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-a6909911-2252-4ea5-bc59-af892a635ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:11.014928 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:11.048903 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 657/1330] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:36:10 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:36:11.335552 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f198b56-3140-4bea-9abd-c4a6f7a9e57a None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:11.337272 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-3f198b56-3140-4bea-9abd-c4a6f7a9e57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:11.445217 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f198b56-3140-4bea-9abd-c4a6f7a9e57a None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:11.465806 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:11.538921 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a148b2-1c0c-4f9e-aaf3-53de2a864e09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:11.539535 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a148b2-1c0c-4f9e-aaf3-53de2a864e09 service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:11.539819 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a148b2-1c0c-4f9e-aaf3-53de2a864e09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:11.540131 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a148b2-1c0c-4f9e-aaf3-53de2a864e09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:11.541762 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-73a148b2-1c0c-4f9e-aaf3-53de2a864e09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:11.658268 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a148b2-1c0c-4f9e-aaf3-53de2a864e09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:11.660236 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a148b2-1c0c-4f9e-aaf3-53de2a864e09 service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:11.679138 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 674/1331] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:36:11 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:36:14.167900 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98103732-19e3-4070-800d-e3b8876bbaab None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:14.168838 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98103732-19e3-4070-800d-e3b8876bbaab None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:14.168838 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98103732-19e3-4070-800d-e3b8876bbaab None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:14.169891 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 658/1332] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:36:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:36:14.179261 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027d8c9-b7da-459e-aff9-6614f0d81370 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:14.179748 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027d8c9-b7da-459e-aff9-6614f0d81370 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:14.179839 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027d8c9-b7da-459e-aff9-6614f0d81370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:14.244341 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:14.247213 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-3027d8c9-b7da-459e-aff9-6614f0d81370 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:36:14.248501 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-3027d8c9-b7da-459e-aff9-6614f0d81370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:14.344580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 675/1333] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:36:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:36:14.629199 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98103732-19e3-4070-800d-e3b8876bbaab None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:14.630789 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-98103732-19e3-4070-800d-e3b8876bbaab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:14.739799 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98103732-19e3-4070-800d-e3b8876bbaab None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:14.813984 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7a1f75-46d1-4887-a52b-144efddbe5f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:14.814538 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a1f75-46d1-4887-a52b-144efddbe5f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:14.814795 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a1f75-46d1-4887-a52b-144efddbe5f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:14.815072 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a1f75-46d1-4887-a52b-144efddbe5f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:14.817402 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-cb7a1f75-46d1-4887-a52b-144efddbe5f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:14.846299 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:14.930024 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7a1f75-46d1-4887-a52b-144efddbe5f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:14.931464 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7a1f75-46d1-4887-a52b-144efddbe5f8 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:14.949392 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 659/1334] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:36:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:36:16.931492 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63ef6f-ad75-4dc4-9a7e-3bffe9247347 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:16.932051 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63ef6f-ad75-4dc4-9a7e-3bffe9247347 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:16.932143 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63ef6f-ad75-4dc4-9a7e-3bffe9247347 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:16.933462 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 676/1335] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:36:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:36:16.938628 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f1a7b-4993-4b0e-a587-675b7db728a8 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:16.939082 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f1a7b-4993-4b0e-a587-675b7db728a8 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:16.939175 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f1a7b-4993-4b0e-a587-675b7db728a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:17.007535 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-513f1a7b-4993-4b0e-a587-675b7db728a8 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:36:17.008796 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-513f1a7b-4993-4b0e-a587-675b7db728a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:17.087377 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:17.110213 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 660/1336] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:36:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:36:17.405173 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a63ef6f-ad75-4dc4-9a7e-3bffe9247347 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:17.406714 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-8a63ef6f-ad75-4dc4-9a7e-3bffe9247347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:17.511176 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a63ef6f-ad75-4dc4-9a7e-3bffe9247347 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:17.540081 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:17.585079 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dfd171-846d-41f6-92fb-6ba7878acd45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:17.585682 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dfd171-846d-41f6-92fb-6ba7878acd45 service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:17.585962 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dfd171-846d-41f6-92fb-6ba7878acd45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:17.586237 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dfd171-846d-41f6-92fb-6ba7878acd45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:17.587812 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-16dfd171-846d-41f6-92fb-6ba7878acd45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:17.833956 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dfd171-846d-41f6-92fb-6ba7878acd45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:17.835288 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dfd171-846d-41f6-92fb-6ba7878acd45 service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:17.853271 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 677/1337] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:36:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:36:20.481427 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c759cff-cd31-4bcc-9f12-572b8fc4bad6 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:20.481944 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c759cff-cd31-4bcc-9f12-572b8fc4bad6 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:20.481944 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c759cff-cd31-4bcc-9f12-572b8fc4bad6 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:20.483126 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 661/1338] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:36:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:36:20.488951 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a78d2-138e-49d4-94cc-57d33669eda8 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:20.489356 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a78d2-138e-49d4-94cc-57d33669eda8 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:20.489499 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a78d2-138e-49d4-94cc-57d33669eda8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:20.555652 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-229a78d2-138e-49d4-94cc-57d33669eda8 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:36:20.556927 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-229a78d2-138e-49d4-94cc-57d33669eda8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:20.565237 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:20.663682 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 678/1339] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:36:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:36:20.953327 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c759cff-cd31-4bcc-9f12-572b8fc4bad6 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:20.954962 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-7c759cff-cd31-4bcc-9f12-572b8fc4bad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:21.063207 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c759cff-cd31-4bcc-9f12-572b8fc4bad6 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:21.137344 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad2804-2c71-4c17-807c-4b4ae4500445 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:21.137949 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad2804-2c71-4c17-807c-4b4ae4500445 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:21.138230 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad2804-2c71-4c17-807c-4b4ae4500445 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:21.138508 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad2804-2c71-4c17-807c-4b4ae4500445 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:21.140140 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-01ad2804-2c71-4c17-807c-4b4ae4500445 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:21.177000 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:21.252568 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ad2804-2c71-4c17-807c-4b4ae4500445 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:21.254667 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ad2804-2c71-4c17-807c-4b4ae4500445 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:21.274304 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 662/1340] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:36:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:36:25.662924 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af52e84e-3a35-4dbb-a972-c308a1804a58 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:25.663456 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af52e84e-3a35-4dbb-a972-c308a1804a58 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:25.663587 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af52e84e-3a35-4dbb-a972-c308a1804a58 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:25.665024 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 679/1341] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:36:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:36:25.671227 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e891f-de5b-4e0b-9711-a66fc97a9c58 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:25.672040 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e891f-de5b-4e0b-9711-a66fc97a9c58 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:25.672040 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e891f-de5b-4e0b-9711-a66fc97a9c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:25.749864 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-d03e891f-de5b-4e0b-9711-a66fc97a9c58 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:36:25.751300 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-d03e891f-de5b-4e0b-9711-a66fc97a9c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:25.849023 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:25.861863 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 663/1342] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:36:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:36:26.175339 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af52e84e-3a35-4dbb-a972-c308a1804a58 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:26.176939 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-af52e84e-3a35-4dbb-a972-c308a1804a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:26.308366 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af52e84e-3a35-4dbb-a972-c308a1804a58 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:26.350227 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:26.387414 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c3041-3cf9-4be3-ba59-c5676f22435e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:26.388045 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c3041-3cf9-4be3-ba59-c5676f22435e service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:26.388325 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c3041-3cf9-4be3-ba59-c5676f22435e service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:26.388629 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c3041-3cf9-4be3-ba59-c5676f22435e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:26.390225 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-c80c3041-3cf9-4be3-ba59-c5676f22435e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:26.519372 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80c3041-3cf9-4be3-ba59-c5676f22435e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:26.520870 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80c3041-3cf9-4be3-ba59-c5676f22435e service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:26.540105 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 680/1343] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:36:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:36:29.506302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f065a-34fe-410c-a791-d10d0adbf64c None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:29.506822 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f065a-34fe-410c-a791-d10d0adbf64c None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:29.506822 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f065a-34fe-410c-a791-d10d0adbf64c None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:29.508112 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 664/1344] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:36:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:36:29.513589 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a449e01-a508-4a91-828d-94977f6a1e9d None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:29.513987 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a449e01-a508-4a91-828d-94977f6a1e9d None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:29.514136 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a449e01-a508-4a91-828d-94977f6a1e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:29.584039 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-5a449e01-a508-4a91-828d-94977f6a1e9d None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:36:29.585323 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-5a449e01-a508-4a91-828d-94977f6a1e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:29.603828 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:29.687133 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 681/1345] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:36:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:36:29.970686 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f065a-34fe-410c-a791-d10d0adbf64c None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:29.972827 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-ea5f065a-34fe-410c-a791-d10d0adbf64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:30.082222 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f065a-34fe-410c-a791-d10d0adbf64c None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:30.156610 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa5201c-f7eb-44a7-8370-5d5a2e621205 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:30.157228 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5201c-f7eb-44a7-8370-5d5a2e621205 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:30.157509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5201c-f7eb-44a7-8370-5d5a2e621205 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:30.157791 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5201c-f7eb-44a7-8370-5d5a2e621205 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:30.160368 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-7aa5201c-f7eb-44a7-8370-5d5a2e621205 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:30.215143 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:30.280773 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa5201c-f7eb-44a7-8370-5d5a2e621205 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:30.282252 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa5201c-f7eb-44a7-8370-5d5a2e621205 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:30.305299 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 665/1346] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:36:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:36:34.625654 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547d90f-0049-4bc4-a313-b75b7c5b68ff None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:34.626186 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547d90f-0049-4bc4-a313-b75b7c5b68ff None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:34.626186 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547d90f-0049-4bc4-a313-b75b7c5b68ff None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:34.628301 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 682/1347] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:36:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:36:34.634096 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3f15a-0d2c-4a29-95c7-2cd6446f3553 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:34.634591 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3f15a-0d2c-4a29-95c7-2cd6446f3553 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:34.634709 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3f15a-0d2c-4a29-95c7-2cd6446f3553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:34.714718 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-eba3f15a-0d2c-4a29-95c7-2cd6446f3553 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:36:34.715977 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-eba3f15a-0d2c-4a29-95c7-2cd6446f3553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:34.842585 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:34.845638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 666/1348] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:36:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:36:35.135444 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d547d90f-0049-4bc4-a313-b75b7c5b68ff None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:35.137025 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-d547d90f-0049-4bc4-a313-b75b7c5b68ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:35.251362 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d547d90f-0049-4bc4-a313-b75b7c5b68ff None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:35.299903 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:35.326823 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43cf4f2-7162-4872-b17d-d787a7522a71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:35.327489 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43cf4f2-7162-4872-b17d-d787a7522a71 service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:35.327991 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43cf4f2-7162-4872-b17d-d787a7522a71 service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:35.328467 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43cf4f2-7162-4872-b17d-d787a7522a71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:35.330366 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-c43cf4f2-7162-4872-b17d-d787a7522a71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:35.443212 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43cf4f2-7162-4872-b17d-d787a7522a71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:35.444679 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43cf4f2-7162-4872-b17d-d787a7522a71 service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:35.463279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 683/1349] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:36:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:36:40.474750 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc552713-a362-43cd-a3bd-de4a40b26586 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:40.475951 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc552713-a362-43cd-a3bd-de4a40b26586 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:40.476497 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc552713-a362-43cd-a3bd-de4a40b26586 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:40.478411 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 667/1350] 10.222.0.15 () {54 vars in 921 bytes} [Wed Aug 24 05:36:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:36:40.484303 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd286bbf-8de4-44c8-9e22-8ddc4ca7b278 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:40.485292 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd286bbf-8de4-44c8-9e22-8ddc4ca7b278 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:40.485841 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd286bbf-8de4-44c8-9e22-8ddc4ca7b278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:40.589369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-fd286bbf-8de4-44c8-9e22-8ddc4ca7b278 None None] MFA Rules not processed for user `6778b7c5ad19484b877494f143050405`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:36:40.590685 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-fd286bbf-8de4-44c8-9e22-8ddc4ca7b278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:40.611266 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:40.810990 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 684/1351] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Aug 24 05:36:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:36:45.218304 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf59bd0-99cd-4726-9de7-46579e1df843 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:45.219027 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf59bd0-99cd-4726-9de7-46579e1df843 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:45.219027 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf59bd0-99cd-4726-9de7-46579e1df843 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:45.220493 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 668/1352] 10.222.0.15 () {54 vars in 921 bytes} [Wed Aug 24 05:36:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:36:45.226016 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff10f7eb-be45-45b1-a7c5-c59744c9241c None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:45.226844 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff10f7eb-be45-45b1-a7c5-c59744c9241c None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:45.227329 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff10f7eb-be45-45b1-a7c5-c59744c9241c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:45.308362 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-ff10f7eb-be45-45b1-a7c5-c59744c9241c None None] MFA Rules not processed for user `6778b7c5ad19484b877494f143050405`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:36:45.309706 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-ff10f7eb-be45-45b1-a7c5-c59744c9241c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:45.391416 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:45.431425 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 685/1353] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Aug 24 05:36:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:36:46.862047 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b633872-2eb4-4fab-b96d-53934eb73fe8 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:46.862571 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b633872-2eb4-4fab-b96d-53934eb73fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:46.862571 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b633872-2eb4-4fab-b96d-53934eb73fe8 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:46.863846 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 669/1354] 10.222.0.15 () {54 vars in 921 bytes} [Wed Aug 24 05:36:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:36:46.869352 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf552b2-b432-4304-a137-97f81af120ac None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:46.869923 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf552b2-b432-4304-a137-97f81af120ac None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:46.869923 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf552b2-b432-4304-a137-97f81af120ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:46.950838 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-8bf552b2-b432-4304-a137-97f81af120ac None None] MFA Rules not processed for user `6778b7c5ad19484b877494f143050405`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:36:46.952054 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-8bf552b2-b432-4304-a137-97f81af120ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:47.062656 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 686/1355] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Aug 24 05:36:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:36:50.599360 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e298e-49bc-47b9-bfd2-de9bce9a08c5 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:50.599961 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e298e-49bc-47b9-bfd2-de9bce9a08c5 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:50.599961 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e298e-49bc-47b9-bfd2-de9bce9a08c5 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:50.601424 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 670/1356] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:36:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:36:50.606816 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27a780-c19b-4d98-90ae-abc756c0202e None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:50.607848 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27a780-c19b-4d98-90ae-abc756c0202e None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:50.608301 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27a780-c19b-4d98-90ae-abc756c0202e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:50.668687 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:50.707297 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-ec27a780-c19b-4d98-90ae-abc756c0202e None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:36:50.709701 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-ec27a780-c19b-4d98-90ae-abc756c0202e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:50.848378 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 687/1357] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:36:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:36:51.156722 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbff568-f2ea-446c-a4f5-ff1185daaa64 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:51.157088 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbff568-f2ea-446c-a4f5-ff1185daaa64 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:51.157306 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbff568-f2ea-446c-a4f5-ff1185daaa64 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:51.159187 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 671/1358] 10.222.0.15 () {54 vars in 962 bytes} [Wed Aug 24 05:36:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:36:51.168298 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be654b7-8080-4eaa-943f-e8737a2f77d9 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:51.168746 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be654b7-8080-4eaa-943f-e8737a2f77d9 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:51.169473 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be654b7-8080-4eaa-943f-e8737a2f77d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:51.267429 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-4be654b7-8080-4eaa-943f-e8737a2f77d9 None None] MFA Rules not processed for user `fbecffb800584c5786813accd061421b`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:36:51.269062 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-4be654b7-8080-4eaa-943f-e8737a2f77d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:51.331041 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:51.419717 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 688/1359] 10.222.0.15 () {58 vars in 1059 bytes} [Wed Aug 24 05:36:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:36:51.420491 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbff568-f2ea-446c-a4f5-ff1185daaa64 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:51.422194 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-ddbff568-f2ea-446c-a4f5-ff1185daaa64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:51.582852 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbff568-f2ea-446c-a4f5-ff1185daaa64 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:51.671439 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:51.714600 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723273c6-1675-4a72-9ad1-59fee50d8cac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:51.715356 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723273c6-1675-4a72-9ad1-59fee50d8cac service nova] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:51.715736 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723273c6-1675-4a72-9ad1-59fee50d8cac service nova] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:51.716112 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723273c6-1675-4a72-9ad1-59fee50d8cac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:51.717862 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-723273c6-1675-4a72-9ad1-59fee50d8cac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:51.812149 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723273c6-1675-4a72-9ad1-59fee50d8cac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:51.813936 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723273c6-1675-4a72-9ad1-59fee50d8cac service nova] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:51.833371 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 672/1360] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:36:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:36:54.007240 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ceef1-db3e-4405-9fee-8fd0ca4766f2 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:54.007791 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ceef1-db3e-4405-9fee-8fd0ca4766f2 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:54.007791 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ceef1-db3e-4405-9fee-8fd0ca4766f2 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:54.009368 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 689/1361] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:36:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:36:54.014799 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01ef86-a406-495e-8b6b-a1593caee619 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:54.015609 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01ef86-a406-495e-8b6b-a1593caee619 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:54.015891 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01ef86-a406-495e-8b6b-a1593caee619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:54.086760 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-5f01ef86-a406-495e-8b6b-a1593caee619 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:36:54.087964 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-5f01ef86-a406-495e-8b6b-a1593caee619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:54.127571 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:54.200447 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 673/1362] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:36:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:36:54.508441 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735ceef1-db3e-4405-9fee-8fd0ca4766f2 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:54.511255 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-735ceef1-db3e-4405-9fee-8fd0ca4766f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:54.691028 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735ceef1-db3e-4405-9fee-8fd0ca4766f2 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:54.701201 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:54.816893 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb4cb56-0f07-49b1-8305-0c084c5b2593 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:54.817519 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4cb56-0f07-49b1-8305-0c084c5b2593 service nova] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:54.817799 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4cb56-0f07-49b1-8305-0c084c5b2593 service nova] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:54.818074 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4cb56-0f07-49b1-8305-0c084c5b2593 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:54.819723 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-7cb4cb56-0f07-49b1-8305-0c084c5b2593 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:54.927962 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb4cb56-0f07-49b1-8305-0c084c5b2593 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:54.929674 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb4cb56-0f07-49b1-8305-0c084c5b2593 service nova] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:54.949349 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 690/1363] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:36:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:36:57.364160 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c22d97-7de4-413f-9ff9-4a023a75d3b4 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:57.364775 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c22d97-7de4-413f-9ff9-4a023a75d3b4 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:57.364775 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c22d97-7de4-413f-9ff9-4a023a75d3b4 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:57.366394 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 674/1364] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:36:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:36:57.371722 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03df23aa-bfdd-4979-b671-75433b8f9d97 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:57.372387 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03df23aa-bfdd-4979-b671-75433b8f9d97 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:57.372387 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03df23aa-bfdd-4979-b671-75433b8f9d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:57.388735 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:57.442256 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-03df23aa-bfdd-4979-b671-75433b8f9d97 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:36:57.443385 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-03df23aa-bfdd-4979-b671-75433b8f9d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:57.546226 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 691/1365] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:36:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:36:57.837343 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59557923-beb5-4674-8108-40b146b62c39 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:57.837957 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59557923-beb5-4674-8108-40b146b62c39 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:57.838322 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59557923-beb5-4674-8108-40b146b62c39 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:57.839983 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 675/1366] 10.222.0.15 () {54 vars in 962 bytes} [Wed Aug 24 05:36:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:36:57.846285 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dbe759-6252-42da-99d5-1b9a4448e147 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:57.848184 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dbe759-6252-42da-99d5-1b9a4448e147 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:57.848184 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dbe759-6252-42da-99d5-1b9a4448e147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:57.930411 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-d8dbe759-6252-42da-99d5-1b9a4448e147 None None] MFA Rules not processed for user `fbecffb800584c5786813accd061421b`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:36:57.931612 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-d8dbe759-6252-42da-99d5-1b9a4448e147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:57.969951 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:58.057241 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 692/1367] 10.222.0.15 () {58 vars in 1059 bytes} [Wed Aug 24 05:36:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:36:58.065169 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59557923-beb5-4674-8108-40b146b62c39 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:58.066809 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-59557923-beb5-4674-8108-40b146b62c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:58.255521 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59557923-beb5-4674-8108-40b146b62c39 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:58.279107 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:58.381473 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe3d0d9-1788-436c-84d4-bd788e0068a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:58.382118 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3d0d9-1788-436c-84d4-bd788e0068a0 service nova] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:58.382355 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3d0d9-1788-436c-84d4-bd788e0068a0 service nova] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:58.382624 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3d0d9-1788-436c-84d4-bd788e0068a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:58.384228 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-6fe3d0d9-1788-436c-84d4-bd788e0068a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:58.485030 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe3d0d9-1788-436c-84d4-bd788e0068a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:58.486990 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe3d0d9-1788-436c-84d4-bd788e0068a0 service nova] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:58.513225 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 676/1368] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:36:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:36:59.013659 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080a3c7-57cb-429f-b6df-bf3cf3934d8d None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:59.014527 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080a3c7-57cb-429f-b6df-bf3cf3934d8d None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:59.014527 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080a3c7-57cb-429f-b6df-bf3cf3934d8d None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:59.016122 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 693/1369] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:36:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:36:59.022422 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08841f70-c965-43fb-a434-e27de02fabea None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:59.022823 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08841f70-c965-43fb-a434-e27de02fabea None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:59.023202 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08841f70-c965-43fb-a434-e27de02fabea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:59.058296 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:59.098872 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-08841f70-c965-43fb-a434-e27de02fabea None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:36:59.100808 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-08841f70-c965-43fb-a434-e27de02fabea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:59.218294 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 677/1370] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:36:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:36:59.816756 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2276beba-6459-425e-b322-e260cfaf3099 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:59.818021 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2276beba-6459-425e-b322-e260cfaf3099 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:59.818021 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2276beba-6459-425e-b322-e260cfaf3099 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:59.819050 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 694/1371] 10.222.0.15 () {54 vars in 964 bytes} [Wed Aug 24 05:36:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:36:59.824381 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c360a94-a950-47f1-a803-52ae3c4c0270 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:59.824937 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c360a94-a950-47f1-a803-52ae3c4c0270 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:59.825100 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c360a94-a950-47f1-a803-52ae3c4c0270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:59.910347 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-4c360a94-a950-47f1-a803-52ae3c4c0270 None None] MFA Rules not processed for user `47bfd618ef39498f91fe7731cd40253b`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:36:59.911565 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-4c360a94-a950-47f1-a803-52ae3c4c0270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:59.966352 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:00.000507 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 678/1372] 10.222.0.15 () {58 vars in 1061 bytes} [Wed Aug 24 05:36:59 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:37:00.008864 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2276beba-6459-425e-b322-e260cfaf3099 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:37:00.010454 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-2276beba-6459-425e-b322-e260cfaf3099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:00.130535 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2276beba-6459-425e-b322-e260cfaf3099 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:37:00.177960 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:00.218061 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597a6638-9f13-4c6e-8cf4-7dea16c33bce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:37:00.219000 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a6638-9f13-4c6e-8cf4-7dea16c33bce service cinder] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:00.219685 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a6638-9f13-4c6e-8cf4-7dea16c33bce service cinder] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:00.220382 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a6638-9f13-4c6e-8cf4-7dea16c33bce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:00.222683 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-597a6638-9f13-4c6e-8cf4-7dea16c33bce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:00.344929 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597a6638-9f13-4c6e-8cf4-7dea16c33bce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:37:00.344929 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597a6638-9f13-4c6e-8cf4-7dea16c33bce service cinder] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:37:00.367908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 695/1373] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:37:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:37:00.712681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1167fd-4757-4267-8551-36f52a9a449e None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:00.712958 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1167fd-4757-4267-8551-36f52a9a449e None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:00.713240 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1167fd-4757-4267-8551-36f52a9a449e None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:00.715031 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 679/1374] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:37:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:37:00.720364 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297c5c9-2cc8-4670-afb7-18d10e70fa5a None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:00.721772 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297c5c9-2cc8-4670-afb7-18d10e70fa5a None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:00.721772 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297c5c9-2cc8-4670-afb7-18d10e70fa5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:00.788649 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-a297c5c9-2cc8-4670-afb7-18d10e70fa5a None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:37:00.789885 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-a297c5c9-2cc8-4670-afb7-18d10e70fa5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:00.811528 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:00.893109 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 696/1375] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:37:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:37:01.193296 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1167fd-4757-4267-8551-36f52a9a449e None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:37:01.194813 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-3e1167fd-4757-4267-8551-36f52a9a449e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:01.345188 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1167fd-4757-4267-8551-36f52a9a449e None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:37:01.402768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:01.474940 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cac6e8-b0bf-4492-ab15-cafa32bc90a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:37:01.475540 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cac6e8-b0bf-4492-ab15-cafa32bc90a4 service nova] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:01.475816 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cac6e8-b0bf-4492-ab15-cafa32bc90a4 service nova] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:01.476087 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cac6e8-b0bf-4492-ab15-cafa32bc90a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:01.477724 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-a7cac6e8-b0bf-4492-ab15-cafa32bc90a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:01.568182 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cac6e8-b0bf-4492-ab15-cafa32bc90a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:37:01.569797 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cac6e8-b0bf-4492-ab15-cafa32bc90a4 service nova] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:37:01.587966 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 680/1376] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:37:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:37:03.704443 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148baeb-b770-4e07-b711-acd494477304 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:03.705005 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148baeb-b770-4e07-b711-acd494477304 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:03.705005 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148baeb-b770-4e07-b711-acd494477304 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:03.706358 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 697/1377] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:37:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:37:03.712441 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbdd213-19bb-4aed-998f-9aa7c47465a4 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:03.713047 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbdd213-19bb-4aed-998f-9aa7c47465a4 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:03.713512 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbdd213-19bb-4aed-998f-9aa7c47465a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:03.785190 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-adbdd213-19bb-4aed-998f-9aa7c47465a4 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:37:03.786411 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-adbdd213-19bb-4aed-998f-9aa7c47465a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:03.793548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:03.890055 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 681/1378] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:37:04.212436 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9148baeb-b770-4e07-b711-acd494477304 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:37:04.213948 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-9148baeb-b770-4e07-b711-acd494477304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:04.372440 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9148baeb-b770-4e07-b711-acd494477304 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:37:04.377819 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:04.496297 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff21abf-7c8b-4031-bb7f-519755f7a010 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:37:04.497243 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff21abf-7c8b-4031-bb7f-519755f7a010 service nova] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:04.497526 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff21abf-7c8b-4031-bb7f-519755f7a010 service nova] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:04.497801 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff21abf-7c8b-4031-bb7f-519755f7a010 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:04.499416 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-aff21abf-7c8b-4031-bb7f-519755f7a010 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:04.612224 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff21abf-7c8b-4031-bb7f-519755f7a010 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:37:04.613851 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff21abf-7c8b-4031-bb7f-519755f7a010 service nova] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:37:04.636960 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 698/1379] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:37:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:37:06.715177 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b80c2-c0ef-43fe-b72a-3216e4080ced None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:06.716026 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b80c2-c0ef-43fe-b72a-3216e4080ced None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:06.716545 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b80c2-c0ef-43fe-b72a-3216e4080ced None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:06.718799 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 682/1380] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:37:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:37:06.724437 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb95bdc5-2e23-4686-91c2-4b8914b5d5e9 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:06.724942 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb95bdc5-2e23-4686-91c2-4b8914b5d5e9 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:06.725260 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb95bdc5-2e23-4686-91c2-4b8914b5d5e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:06.736883 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:06.799462 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-fb95bdc5-2e23-4686-91c2-4b8914b5d5e9 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:37:06.800864 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-fb95bdc5-2e23-4686-91c2-4b8914b5d5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:06.905052 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 699/1381] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:37:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:37:07.214293 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b80c2-c0ef-43fe-b72a-3216e4080ced None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:37:07.215904 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-3a2b80c2-c0ef-43fe-b72a-3216e4080ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:07.379887 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:07.392870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b80c2-c0ef-43fe-b72a-3216e4080ced None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:37:07.525032 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d89dc5a-df89-4777-ac45-f7fbe542dc6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:37:07.525638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89dc5a-df89-4777-ac45-f7fbe542dc6f service nova] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:07.525914 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89dc5a-df89-4777-ac45-f7fbe542dc6f service nova] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:07.526184 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89dc5a-df89-4777-ac45-f7fbe542dc6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:07.527833 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-5d89dc5a-df89-4777-ac45-f7fbe542dc6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:07.625657 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d89dc5a-df89-4777-ac45-f7fbe542dc6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:37:07.627300 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d89dc5a-df89-4777-ac45-f7fbe542dc6f service nova] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:37:07.638846 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:07.648045 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 683/1382] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:37:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:37:10.050170 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c809e-57dc-4826-af51-318eca4c749d None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:10.050709 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c809e-57dc-4826-af51-318eca4c749d None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:10.050709 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c809e-57dc-4826-af51-318eca4c749d None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:10.052171 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 700/1383] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:37:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:37:10.058730 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5fcc1-40ad-4917-bf77-dac45a964ac5 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:10.059202 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5fcc1-40ad-4917-bf77-dac45a964ac5 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:10.059687 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5fcc1-40ad-4917-bf77-dac45a964ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:10.136413 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-6df5fcc1-40ad-4917-bf77-dac45a964ac5 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:37:10.137730 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-6df5fcc1-40ad-4917-bf77-dac45a964ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:10.248925 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 684/1384] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:37:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:37:10.580764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c809e-57dc-4826-af51-318eca4c749d None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:37:10.582249 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-d15c809e-57dc-4826-af51-318eca4c749d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:10.685207 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:10.764299 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c809e-57dc-4826-af51-318eca4c749d None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:37:10.902392 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ac3a48-853d-4447-8c6d-edd3b18f5aca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:37:10.903007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac3a48-853d-4447-8c6d-edd3b18f5aca service nova] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:10.903288 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac3a48-853d-4447-8c6d-edd3b18f5aca service nova] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:10.903561 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac3a48-853d-4447-8c6d-edd3b18f5aca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:10.905236 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-66ac3a48-853d-4447-8c6d-edd3b18f5aca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:10.962698 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:11.010374 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ac3a48-853d-4447-8c6d-edd3b18f5aca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:37:11.012189 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ac3a48-853d-4447-8c6d-edd3b18f5aca service nova] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:37:11.037249 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 701/1385] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:37:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:37:13.114653 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc70d431-1768-463f-b1c6-0a3046df2202 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:13.115222 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc70d431-1768-463f-b1c6-0a3046df2202 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:13.115222 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc70d431-1768-463f-b1c6-0a3046df2202 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:13.116547 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 685/1386] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:37:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:37:13.122167 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd52ef8-1882-4c8a-ac55-e280a25b8bce None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:13.122837 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd52ef8-1882-4c8a-ac55-e280a25b8bce None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:13.122837 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd52ef8-1882-4c8a-ac55-e280a25b8bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:13.194163 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-6dd52ef8-1882-4c8a-ac55-e280a25b8bce None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:37:13.195399 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-6dd52ef8-1882-4c8a-ac55-e280a25b8bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:13.302072 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 702/1387] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:37:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:37:13.597103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc70d431-1768-463f-b1c6-0a3046df2202 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:37:13.598836 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-cc70d431-1768-463f-b1c6-0a3046df2202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:13.691507 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:13.789458 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc70d431-1768-463f-b1c6-0a3046df2202 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:37:13.946887 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd2a0f-c6f7-47d3-9b51-a4dcd212d2ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:37:13.947913 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd2a0f-c6f7-47d3-9b51-a4dcd212d2ca service nova] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:13.948342 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd2a0f-c6f7-47d3-9b51-a4dcd212d2ca service nova] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:13.948998 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd2a0f-c6f7-47d3-9b51-a4dcd212d2ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:13.951452 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-1dcd2a0f-c6f7-47d3-9b51-a4dcd212d2ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:14.011076 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:14.071259 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd2a0f-c6f7-47d3-9b51-a4dcd212d2ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:37:14.072984 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd2a0f-c6f7-47d3-9b51-a4dcd212d2ca service nova] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:37:14.093338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 686/1388] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:37:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:37:16.226823 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64615f02-ea74-4e01-8fe4-3d22772c02aa None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:16.227409 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64615f02-ea74-4e01-8fe4-3d22772c02aa None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:16.227409 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64615f02-ea74-4e01-8fe4-3d22772c02aa None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:16.229186 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 703/1389] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:37:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:37:16.235657 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5966d5-31ef-4c32-ab09-05d3f88774db None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:16.236328 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5966d5-31ef-4c32-ab09-05d3f88774db None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:16.236732 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5966d5-31ef-4c32-ab09-05d3f88774db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:16.280249 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35aef9d-0dc2-4111-b01f-d49d9def9384 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:16.280584 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35aef9d-0dc2-4111-b01f-d49d9def9384 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:16.280868 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35aef9d-0dc2-4111-b01f-d49d9def9384 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:16.282670 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 704/1390] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:37:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:37:16.288520 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b5ab8-56d8-4fe7-a420-f689a69fc986 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:16.288812 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b5ab8-56d8-4fe7-a420-f689a69fc986 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:16.289129 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b5ab8-56d8-4fe7-a420-f689a69fc986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:16.320373 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-ba5966d5-31ef-4c32-ab09-05d3f88774db None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:37:16.321631 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-ba5966d5-31ef-4c32-ab09-05d3f88774db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:16.328445 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:16.366865 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-438b5ab8-56d8-4fe7-a420-f689a69fc986 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:37:16.368080 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-438b5ab8-56d8-4fe7-a420-f689a69fc986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:16.426294 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 687/1391] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:37:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:37:16.469889 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 705/1392] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:37:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:37:16.729129 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5966d5-31ef-4c32-ab09-05d3f88774db None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:37:16.730301 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-ba5966d5-31ef-4c32-ab09-05d3f88774db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:16.745389 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:16.776880 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2291d1b-bf3a-41af-b505-eb2aef329522 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:16.777167 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2291d1b-bf3a-41af-b505-eb2aef329522 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:16.777454 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2291d1b-bf3a-41af-b505-eb2aef329522 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:16.778859 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 706/1393] 10.222.0.15 () {54 vars in 967 bytes} [Wed Aug 24 05:37:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:37:16.784153 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177c620-c388-4d67-a2f1-80f845818c48 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:16.789791 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177c620-c388-4d67-a2f1-80f845818c48 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:16.789791 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177c620-c388-4d67-a2f1-80f845818c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:16.869724 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5966d5-31ef-4c32-ab09-05d3f88774db None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:37:16.873339 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-7177c620-c388-4d67-a2f1-80f845818c48 None None] MFA Rules not processed for user `9aee9c524a974a248c3bf363444f584a`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:37:16.874597 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-7177c620-c388-4d67-a2f1-80f845818c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:16.928522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:16.957714 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 707/1394] 10.222.0.15 () {58 vars in 1064 bytes} [Wed Aug 24 05:37:16 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:37:16.965260 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7177c620-c388-4d67-a2f1-80f845818c48 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:37:16.973348 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7177c620-c388-4d67-a2f1-80f845818c48 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:37:16.989487 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5772d0-89c1-4777-b116-26869c661b75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:37:16.990026 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5772d0-89c1-4777-b116-26869c661b75 service nova] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:16.990312 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5772d0-89c1-4777-b116-26869c661b75 service nova] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:16.990585 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5772d0-89c1-4777-b116-26869c661b75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:16.992292 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-3b5772d0-89c1-4777-b116-26869c661b75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:17.000970 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:17.049842 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e3e36-724e-40d9-a365-74e1b7ebb290 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9aee9c524a974a248c3bf363444f584a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:37:17.050456 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e3e36-724e-40d9-a365-74e1b7ebb290 service glance] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:17.050735 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e3e36-724e-40d9-a365-74e1b7ebb290 service glance] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:17.051011 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e3e36-724e-40d9-a365-74e1b7ebb290 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:17.052709 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-c72e3e36-724e-40d9-a365-74e1b7ebb290 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:17.087152 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5772d0-89c1-4777-b116-26869c661b75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:37:17.088817 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5772d0-89c1-4777-b116-26869c661b75 service nova] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:37:17.108834 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 688/1395] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:37:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:37:17.166279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72e3e36-724e-40d9-a365-74e1b7ebb290 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:37:17.167725 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72e3e36-724e-40d9-a365-74e1b7ebb290 service glance] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:37:17.182946 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:17.187463 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 708/1396] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:37:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:37:18.104026 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff54bab3-d38f-47eb-8bae-c08402da5d15 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:18.104574 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff54bab3-d38f-47eb-8bae-c08402da5d15 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:18.104694 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff54bab3-d38f-47eb-8bae-c08402da5d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:18.173499 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-ff54bab3-d38f-47eb-8bae-c08402da5d15 None None] MFA Rules not processed for user `604380c191c8461bbb937b8d951238bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:37:18.174728 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-ff54bab3-d38f-47eb-8bae-c08402da5d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:18.262542 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:18.266015 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 689/1397] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Aug 24 05:37:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:37:18.309336 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b657d-05f3-49ff-9a2b-18c91989b6a4 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:18.331983 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b657d-05f3-49ff-9a2b-18c91989b6a4 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:18.331983 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b657d-05f3-49ff-9a2b-18c91989b6a4 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:18.331983 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 709/1398] 10.222.0.15 () {54 vars in 943 bytes} [Wed Aug 24 05:37:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:37:18.332822 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af6fc0-83e0-49d1-8d31-f5ac67d3f007 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:18.332822 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af6fc0-83e0-49d1-8d31-f5ac67d3f007 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:18.332822 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af6fc0-83e0-49d1-8d31-f5ac67d3f007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:18.400560 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-99af6fc0-83e0-49d1-8d31-f5ac67d3f007 None None] MFA Rules not processed for user `bb2b6cae4a004539a95fc7dd558db255`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:37:18.401814 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-99af6fc0-83e0-49d1-8d31-f5ac67d3f007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:18.513457 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 690/1399] 10.222.0.15 () {58 vars in 1040 bytes} [Wed Aug 24 05:37:18 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:37:18.523414 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b657d-05f3-49ff-9a2b-18c91989b6a4 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:37:18.525112 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-aa9b657d-05f3-49ff-9a2b-18c91989b6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:18.668363 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b657d-05f3-49ff-9a2b-18c91989b6a4 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:37:18.756266 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:18.778055 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b63d9-4cc1-414c-bc2a-e28fba5dcd86 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb2b6cae4a004539a95fc7dd558db255', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:37:18.778973 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b63d9-4cc1-414c-bc2a-e28fba5dcd86 service swift] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:18.779421 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b63d9-4cc1-414c-bc2a-e28fba5dcd86 service swift] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:18.779949 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b63d9-4cc1-414c-bc2a-e28fba5dcd86 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:18.781751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-db5b63d9-4cc1-414c-bc2a-e28fba5dcd86 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:18.875035 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b63d9-4cc1-414c-bc2a-e28fba5dcd86 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:37:18.877936 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b63d9-4cc1-414c-bc2a-e28fba5dcd86 service swift] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:37:18.886410 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 710/1400] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Aug 24 05:37:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 05:37:19.272075 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144e4f3-f593-4d16-b291-9910abd0faa1 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:19.272572 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144e4f3-f593-4d16-b291-9910abd0faa1 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:19.272686 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144e4f3-f593-4d16-b291-9910abd0faa1 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:19.274288 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 691/1401] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:37:19.279747 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43704319-b946-46e6-a143-06d7b7a85359 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:19.280039 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43704319-b946-46e6-a143-06d7b7a85359 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:19.280317 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43704319-b946-46e6-a143-06d7b7a85359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:19.357118 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-43704319-b946-46e6-a143-06d7b7a85359 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:37:19.358473 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-43704319-b946-46e6-a143-06d7b7a85359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:19.433371 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:19.472441 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 711/1402] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:37:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:37:19.781591 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144e4f3-f593-4d16-b291-9910abd0faa1 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:37:19.788814 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-e144e4f3-f593-4d16-b291-9910abd0faa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:19.864419 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:19.964839 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144e4f3-f593-4d16-b291-9910abd0faa1 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:37:20.096053 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e1cfe-fe94-4a57-9aed-03760591a1db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:37:20.096958 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e1cfe-fe94-4a57-9aed-03760591a1db service nova] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:20.097412 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e1cfe-fe94-4a57-9aed-03760591a1db service nova] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:20.097840 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e1cfe-fe94-4a57-9aed-03760591a1db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:20.099600 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-044e1cfe-fe94-4a57-9aed-03760591a1db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:20.118365 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:20.206157 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044e1cfe-fe94-4a57-9aed-03760591a1db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:37:20.207831 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044e1cfe-fe94-4a57-9aed-03760591a1db service nova] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:37:20.232966 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 692/1403] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:37:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:37:22.425271 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5c249-d899-4463-8200-7d69c3c84e7a None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:22.425875 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5c249-d899-4463-8200-7d69c3c84e7a None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:22.425875 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5c249-d899-4463-8200-7d69c3c84e7a None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:22.428787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 712/1404] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:37:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:37:22.432643 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657aff4-f9b7-4d8b-9c79-92dd0c95c66e None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:22.433117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657aff4-f9b7-4d8b-9c79-92dd0c95c66e None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:22.433237 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657aff4-f9b7-4d8b-9c79-92dd0c95c66e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:22.519405 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-9657aff4-f9b7-4d8b-9c79-92dd0c95c66e None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:37:22.520678 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-9657aff4-f9b7-4d8b-9c79-92dd0c95c66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:22.640280 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:22.648258 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 693/1405] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:37:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:37:22.965521 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d5c249-d899-4463-8200-7d69c3c84e7a None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:37:22.967076 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-36d5c249-d899-4463-8200-7d69c3c84e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:23.134584 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d5c249-d899-4463-8200-7d69c3c84e7a None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:37:23.196165 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:23.263907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d89123-dae9-4b7b-929f-95054d25799b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:37:23.264555 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d89123-dae9-4b7b-929f-95054d25799b service nova] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:23.264841 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d89123-dae9-4b7b-929f-95054d25799b service nova] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:23.265120 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d89123-dae9-4b7b-929f-95054d25799b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:23.268687 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-42d89123-dae9-4b7b-929f-95054d25799b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:23.371529 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d89123-dae9-4b7b-929f-95054d25799b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:37:23.373384 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d89123-dae9-4b7b-929f-95054d25799b service nova] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:37:23.396511 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 713/1406] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:37:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:37:25.477733 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe882c1-b160-45ea-b49c-82201970734b None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:25.478246 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe882c1-b160-45ea-b49c-82201970734b None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:25.478246 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe882c1-b160-45ea-b49c-82201970734b None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:25.479550 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 694/1407] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:37:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:37:25.485821 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704806f-bf8b-4789-8aaa-760faed82803 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:25.486415 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704806f-bf8b-4789-8aaa-760faed82803 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:25.486415 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704806f-bf8b-4789-8aaa-760faed82803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:25.565760 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-5704806f-bf8b-4789-8aaa-760faed82803 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:37:25.567086 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-5704806f-bf8b-4789-8aaa-760faed82803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:25.582177 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:25.689787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 714/1408] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:37:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:37:25.996570 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe882c1-b160-45ea-b49c-82201970734b None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:37:25.998633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-efe882c1-b160-45ea-b49c-82201970734b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:26.172002 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe882c1-b160-45ea-b49c-82201970734b None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:37:26.264879 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:26.322140 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfcc5ef-9764-489b-8a83-b64171c5753e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:37:26.323186 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcc5ef-9764-489b-8a83-b64171c5753e service nova] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:26.323546 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcc5ef-9764-489b-8a83-b64171c5753e service nova] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:26.323892 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcc5ef-9764-489b-8a83-b64171c5753e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:26.325644 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-4dfcc5ef-9764-489b-8a83-b64171c5753e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:26.426819 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfcc5ef-9764-489b-8a83-b64171c5753e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:37:26.428445 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfcc5ef-9764-489b-8a83-b64171c5753e service nova] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:37:26.448005 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 695/1409] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:37:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:37:35.683211 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=38 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:36.445819 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=64 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:43.043384 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116dc7cb-28df-487f-bce8-248227b26c80 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:43.043935 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116dc7cb-28df-487f-bce8-248227b26c80 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:43.044128 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116dc7cb-28df-487f-bce8-248227b26c80 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:43.045521 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 715/1410] 10.222.0.15 () {54 vars in 918 bytes} [Wed Aug 24 05:37:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:37:43.051665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197413b4-f8a6-446a-9f3f-28b358140677 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:43.052249 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197413b4-f8a6-446a-9f3f-28b358140677 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:43.052249 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197413b4-f8a6-446a-9f3f-28b358140677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:43.130950 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-197413b4-f8a6-446a-9f3f-28b358140677 None None] MFA Rules not processed for user `6778b7c5ad19484b877494f143050405`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:37:43.132212 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-197413b4-f8a6-446a-9f3f-28b358140677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:43.241880 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 696/1411] 10.222.0.15 () {58 vars in 1015 bytes} [Wed Aug 24 05:37:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:37:43.257369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea0ded-1cf4-4a45-904a-64d72b260496 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:43.257651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea0ded-1cf4-4a45-904a-64d72b260496 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:43.257925 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea0ded-1cf4-4a45-904a-64d72b260496 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:43.259241 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 716/1412] 10.222.0.15 () {54 vars in 918 bytes} [Wed Aug 24 05:37:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:37:43.264269 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5c8a7-4bdf-4ca4-bf81-e9850b5511b7 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:43.264603 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5c8a7-4bdf-4ca4-bf81-e9850b5511b7 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:43.264892 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5c8a7-4bdf-4ca4-bf81-e9850b5511b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:43.337681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:43.340442 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-f2c5c8a7-4bdf-4ca4-bf81-e9850b5511b7 None None] MFA Rules not processed for user `fbecffb800584c5786813accd061421b`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:37:43.341671 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-f2c5c8a7-4bdf-4ca4-bf81-e9850b5511b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:43.462424 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 697/1413] 10.222.0.15 () {58 vars in 1015 bytes} [Wed Aug 24 05:37:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:37:45.926226 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2dcdf-ad58-4ded-bc36-7eed31527109 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:45.926807 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2dcdf-ad58-4ded-bc36-7eed31527109 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:45.926807 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2dcdf-ad58-4ded-bc36-7eed31527109 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:45.928086 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 717/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 05:37:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:37:45.933651 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3569e-d6d7-48fd-b121-ddec3ea15c87 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:45.934149 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3569e-d6d7-48fd-b121-ddec3ea15c87 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:45.934266 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3569e-d6d7-48fd-b121-ddec3ea15c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:46.118619 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-3dc3569e-d6d7-48fd-b121-ddec3ea15c87 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:37:46.119846 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-3dc3569e-d6d7-48fd-b121-ddec3ea15c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:46.175417 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:46.218115 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 698/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 05:37:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:37:46.508201 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2dcdf-ad58-4ded-bc36-7eed31527109 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:37:46.509791 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-19c2dcdf-ad58-4ded-bc36-7eed31527109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:46.665469 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2dcdf-ad58-4ded-bc36-7eed31527109 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:37:46.744650 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:46.786333 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b7f2e-e33d-4453-8906-02a01ea88927 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:37:46.787247 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b7f2e-e33d-4453-8906-02a01ea88927 service nova] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:46.787508 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b7f2e-e33d-4453-8906-02a01ea88927 service nova] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:46.787784 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b7f2e-e33d-4453-8906-02a01ea88927 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:46.789417 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-225b7f2e-e33d-4453-8906-02a01ea88927 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:46.879918 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225b7f2e-e33d-4453-8906-02a01ea88927 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:37:46.881604 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225b7f2e-e33d-4453-8906-02a01ea88927 service nova] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:37:46.902007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 718/1416] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:37:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:37:47.810586 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ed3af-b8d5-4e48-b543-3160ec9aca18 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:47.811139 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ed3af-b8d5-4e48-b543-3160ec9aca18 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:47.811139 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ed3af-b8d5-4e48-b543-3160ec9aca18 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:47.813077 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 699/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 05:37:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:37:47.818057 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62204a66-72af-40d1-9211-c2d77a882e9b None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:47.818508 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62204a66-72af-40d1-9211-c2d77a882e9b None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:47.818604 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62204a66-72af-40d1-9211-c2d77a882e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:47.896106 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-62204a66-72af-40d1-9211-c2d77a882e9b None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:37:47.897289 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-62204a66-72af-40d1-9211-c2d77a882e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:47.924280 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:47.996560 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 719/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 05:37:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:37:48.151155 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ed3af-b8d5-4e48-b543-3160ec9aca18 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:37:48.152700 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-005ed3af-b8d5-4e48-b543-3160ec9aca18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:48.301724 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ed3af-b8d5-4e48-b543-3160ec9aca18 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:37:48.342098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:48.415867 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525472c6-1dcd-4761-8bbe-c16bef8ebcdf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:37:48.416390 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525472c6-1dcd-4761-8bbe-c16bef8ebcdf service nova] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:48.416698 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525472c6-1dcd-4761-8bbe-c16bef8ebcdf service nova] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:48.416978 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525472c6-1dcd-4761-8bbe-c16bef8ebcdf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:48.418607 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-525472c6-1dcd-4761-8bbe-c16bef8ebcdf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:48.508135 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525472c6-1dcd-4761-8bbe-c16bef8ebcdf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:37:48.509789 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525472c6-1dcd-4761-8bbe-c16bef8ebcdf service nova] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:37:48.527247 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 700/1419] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:37:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:37:49.438681 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa86d7-b321-4678-a118-3c60d641d714 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:49.439196 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa86d7-b321-4678-a118-3c60d641d714 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:49.439196 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa86d7-b321-4678-a118-3c60d641d714 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:49.440571 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 720/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 05:37:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:37:49.445653 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f8ac40-9c5b-49dd-92e9-157353e03ade None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:49.446084 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f8ac40-9c5b-49dd-92e9-157353e03ade None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:49.446189 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f8ac40-9c5b-49dd-92e9-157353e03ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:49.501270 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:49.523988 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-92f8ac40-9c5b-49dd-92e9-157353e03ade None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:37:49.525251 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-92f8ac40-9c5b-49dd-92e9-157353e03ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:49.623759 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 701/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 05:37:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:37:49.778288 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aa86d7-b321-4678-a118-3c60d641d714 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:37:49.779810 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-40aa86d7-b321-4678-a118-3c60d641d714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:49.936794 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aa86d7-b321-4678-a118-3c60d641d714 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:37:49.949401 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:50.051994 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa04c35-19d9-4152-a80c-b4008a289efc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:37:50.052531 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa04c35-19d9-4152-a80c-b4008a289efc service nova] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:50.052810 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa04c35-19d9-4152-a80c-b4008a289efc service nova] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:50.053080 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa04c35-19d9-4152-a80c-b4008a289efc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:50.054662 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-1fa04c35-19d9-4152-a80c-b4008a289efc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:50.144418 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa04c35-19d9-4152-a80c-b4008a289efc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:37:50.146021 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa04c35-19d9-4152-a80c-b4008a289efc service nova] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:37:50.165396 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 721/1422] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:37:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:37:51.073155 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d840e5-1448-48a3-9011-4e057fdd8ddf None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:51.073648 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d840e5-1448-48a3-9011-4e057fdd8ddf None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:51.073745 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d840e5-1448-48a3-9011-4e057fdd8ddf None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:51.075143 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 702/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 05:37:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:37:51.080885 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305bdd59-8bea-4f95-b68a-9c6ac2e38c44 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:51.085764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305bdd59-8bea-4f95-b68a-9c6ac2e38c44 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:51.085764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305bdd59-8bea-4f95-b68a-9c6ac2e38c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:51.102510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:51.164572 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-305bdd59-8bea-4f95-b68a-9c6ac2e38c44 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:37:51.165842 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-305bdd59-8bea-4f95-b68a-9c6ac2e38c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:51.272891 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 722/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 05:37:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:37:51.430897 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d840e5-1448-48a3-9011-4e057fdd8ddf None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:37:51.432925 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-35d840e5-1448-48a3-9011-4e057fdd8ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:51.568671 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:51.617259 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d840e5-1448-48a3-9011-4e057fdd8ddf None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:37:51.747696 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80239816-4617-4cef-a5cb-ebdeb35ae5a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:37:51.748519 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80239816-4617-4cef-a5cb-ebdeb35ae5a7 service nova] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:51.749025 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80239816-4617-4cef-a5cb-ebdeb35ae5a7 service nova] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:51.749561 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80239816-4617-4cef-a5cb-ebdeb35ae5a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:51.752105 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-80239816-4617-4cef-a5cb-ebdeb35ae5a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:51.850382 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:51.866144 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80239816-4617-4cef-a5cb-ebdeb35ae5a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:37:51.867829 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80239816-4617-4cef-a5cb-ebdeb35ae5a7 service nova] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:37:51.888806 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 703/1425] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:37:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:37:52.803861 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c57b1f-395a-44d0-9c7a-e246ce193124 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:52.804360 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c57b1f-395a-44d0-9c7a-e246ce193124 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:52.804488 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c57b1f-395a-44d0-9c7a-e246ce193124 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:52.805915 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 723/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 05:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:37:52.810982 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa2557-3ae7-4631-b8b9-59527348434a None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:52.811397 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa2557-3ae7-4631-b8b9-59527348434a None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:52.811522 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa2557-3ae7-4631-b8b9-59527348434a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:52.892107 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-33aa2557-3ae7-4631-b8b9-59527348434a None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:37:52.893358 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-33aa2557-3ae7-4631-b8b9-59527348434a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:53.002038 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 704/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 05:37:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:37:53.154155 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c57b1f-395a-44d0-9c7a-e246ce193124 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:37:53.155760 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-29c57b1f-395a-44d0-9c7a-e246ce193124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:53.245394 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:53.308354 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c57b1f-395a-44d0-9c7a-e246ce193124 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:37:53.427188 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca610c28-aa6d-43ba-9e02-b121ea9a0738 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:37:53.427705 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca610c28-aa6d-43ba-9e02-b121ea9a0738 service nova] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:53.427981 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca610c28-aa6d-43ba-9e02-b121ea9a0738 service nova] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:53.428252 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca610c28-aa6d-43ba-9e02-b121ea9a0738 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:53.429970 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-ca610c28-aa6d-43ba-9e02-b121ea9a0738 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:53.478256 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:53.520371 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca610c28-aa6d-43ba-9e02-b121ea9a0738 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:37:53.522009 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca610c28-aa6d-43ba-9e02-b121ea9a0738 service nova] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:37:53.540258 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 724/1428] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:37:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:37:54.426587 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c62f96-abdf-4b0c-bcee-65c85ebcd296 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:54.427162 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c62f96-abdf-4b0c-bcee-65c85ebcd296 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:54.427162 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c62f96-abdf-4b0c-bcee-65c85ebcd296 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:54.428935 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 705/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 05:37:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:37:54.433842 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ea843-80e9-41c1-9bad-6020b8e41f69 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:54.434277 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ea843-80e9-41c1-9bad-6020b8e41f69 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:54.434384 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ea843-80e9-41c1-9bad-6020b8e41f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:54.511312 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-e07ea843-80e9-41c1-9bad-6020b8e41f69 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:37:54.512463 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-e07ea843-80e9-41c1-9bad-6020b8e41f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:54.610176 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 725/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 05:37:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:37:54.761982 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c62f96-abdf-4b0c-bcee-65c85ebcd296 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:37:54.763459 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-67c62f96-abdf-4b0c-bcee-65c85ebcd296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:54.813310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:54.910365 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c62f96-abdf-4b0c-bcee-65c85ebcd296 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:37:55.029808 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beaeb11-ad61-4ce0-8d21-136acad15316 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:37:55.030329 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beaeb11-ad61-4ce0-8d21-136acad15316 service nova] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:55.030604 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beaeb11-ad61-4ce0-8d21-136acad15316 service nova] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:55.030874 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beaeb11-ad61-4ce0-8d21-136acad15316 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:55.032616 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-9beaeb11-ad61-4ce0-8d21-136acad15316 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:55.039719 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:55.125488 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beaeb11-ad61-4ce0-8d21-136acad15316 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:37:55.127093 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beaeb11-ad61-4ce0-8d21-136acad15316 service nova] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:37:55.146494 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 706/1431] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:37:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:37:56.031171 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab66db-5688-4ce1-b364-33dcde3707c9 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:56.031902 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab66db-5688-4ce1-b364-33dcde3707c9 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:56.031902 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab66db-5688-4ce1-b364-33dcde3707c9 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:56.033142 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 726/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 05:37:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:37:56.038469 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff020fa3-f766-4d52-a16f-2f040b3a7556 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:56.038874 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff020fa3-f766-4d52-a16f-2f040b3a7556 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:56.039030 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff020fa3-f766-4d52-a16f-2f040b3a7556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:56.116080 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-ff020fa3-f766-4d52-a16f-2f040b3a7556 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:37:56.117321 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-ff020fa3-f766-4d52-a16f-2f040b3a7556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:56.189004 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:56.218411 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 707/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 05:37:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:37:56.370593 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab66db-5688-4ce1-b364-33dcde3707c9 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:37:56.372042 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-ccab66db-5688-4ce1-b364-33dcde3707c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:56.391019 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:56.523315 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab66db-5688-4ce1-b364-33dcde3707c9 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:37:56.614095 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:56.641484 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5210d04a-0152-4727-99cc-4c7def8986e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:37:56.642097 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210d04a-0152-4727-99cc-4c7def8986e4 service nova] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:56.642375 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210d04a-0152-4727-99cc-4c7def8986e4 service nova] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:56.642650 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210d04a-0152-4727-99cc-4c7def8986e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:56.644266 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-5210d04a-0152-4727-99cc-4c7def8986e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:56.743433 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5210d04a-0152-4727-99cc-4c7def8986e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:37:56.745176 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5210d04a-0152-4727-99cc-4c7def8986e4 service nova] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:37:56.763924 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 727/1434] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:37:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:37:57.657994 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd46352-5ea1-4f9c-ac93-5d894bf6d034 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:57.658653 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd46352-5ea1-4f9c-ac93-5d894bf6d034 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:57.658653 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd46352-5ea1-4f9c-ac93-5d894bf6d034 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:57.659921 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 708/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 05:37:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:37:57.665350 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e17867-e6a4-418c-bae6-e84c1c6490d5 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:57.665760 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e17867-e6a4-418c-bae6-e84c1c6490d5 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:57.665944 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e17867-e6a4-418c-bae6-e84c1c6490d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:57.743361 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-75e17867-e6a4-418c-bae6-e84c1c6490d5 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:37:57.744615 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-75e17867-e6a4-418c-bae6-e84c1c6490d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:57.786397 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:57.841425 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 728/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 05:37:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:37:57.997203 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd46352-5ea1-4f9c-ac93-5d894bf6d034 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:37:57.998777 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-6cd46352-5ea1-4f9c-ac93-5d894bf6d034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:58.149018 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd46352-5ea1-4f9c-ac93-5d894bf6d034 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:37:58.205742 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:58.263366 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81a0480-c189-48ff-9853-549dc9e5792a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:37:58.263885 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a0480-c189-48ff-9853-549dc9e5792a service nova] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:58.264157 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a0480-c189-48ff-9853-549dc9e5792a service nova] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:58.264447 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a0480-c189-48ff-9853-549dc9e5792a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:58.266199 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-e81a0480-c189-48ff-9853-549dc9e5792a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:58.354119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81a0480-c189-48ff-9853-549dc9e5792a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:37:58.355693 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81a0480-c189-48ff-9853-549dc9e5792a service nova] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:37:58.374114 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 709/1437] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:37:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:37:59.261927 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e0139-c440-400e-95d5-9dd56a37c318 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:59.262471 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e0139-c440-400e-95d5-9dd56a37c318 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:59.262471 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e0139-c440-400e-95d5-9dd56a37c318 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:59.263898 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 729/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 05:37:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:37:59.269765 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae6156-3b14-476d-891a-13e20825926a None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:59.270203 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae6156-3b14-476d-891a-13e20825926a None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:59.270311 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae6156-3b14-476d-891a-13e20825926a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:59.346872 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-5fae6156-3b14-476d-891a-13e20825926a None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:37:59.348109 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-5fae6156-3b14-476d-891a-13e20825926a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:59.350139 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:59.446294 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 710/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 05:37:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:37:59.598446 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e0139-c440-400e-95d5-9dd56a37c318 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:37:59.599962 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-b14e0139-c440-400e-95d5-9dd56a37c318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:59.745392 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e0139-c440-400e-95d5-9dd56a37c318 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:37:59.775136 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:59.863304 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca9775-0e73-47ef-9104-dcc2c243fc26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:37:59.863907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca9775-0e73-47ef-9104-dcc2c243fc26 service nova] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:37:59.864185 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca9775-0e73-47ef-9104-dcc2c243fc26 service nova] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:37:59.864490 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca9775-0e73-47ef-9104-dcc2c243fc26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:37:59.866109 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-abca9775-0e73-47ef-9104-dcc2c243fc26 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:37:59.963490 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abca9775-0e73-47ef-9104-dcc2c243fc26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:37:59.965139 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abca9775-0e73-47ef-9104-dcc2c243fc26 service nova] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:37:59.983221 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 730/1440] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:37:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:38:01.862872 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42eb1d7-6e4a-48c4-bbbb-8646a25023fa None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:01.863457 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42eb1d7-6e4a-48c4-bbbb-8646a25023fa None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:01.863457 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42eb1d7-6e4a-48c4-bbbb-8646a25023fa None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:01.864812 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 711/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 05:38:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:38:01.870091 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8738aa2f-6f77-46bb-8861-7f16110a2457 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:01.870499 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8738aa2f-6f77-46bb-8861-7f16110a2457 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:01.870634 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8738aa2f-6f77-46bb-8861-7f16110a2457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:01.912542 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:01.938507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-8738aa2f-6f77-46bb-8861-7f16110a2457 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:38:01.939721 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-8738aa2f-6f77-46bb-8861-7f16110a2457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:02.039385 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 731/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 05:38:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:38:02.325441 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42eb1d7-6e4a-48c4-bbbb-8646a25023fa None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:38:02.327049 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-b42eb1d7-6e4a-48c4-bbbb-8646a25023fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:02.431303 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42eb1d7-6e4a-48c4-bbbb-8646a25023fa None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:38:02.463786 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:02.506681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba92f9-1e8d-4d5a-bdcf-31b228b16f3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:38:02.507281 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba92f9-1e8d-4d5a-bdcf-31b228b16f3e service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:02.507558 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba92f9-1e8d-4d5a-bdcf-31b228b16f3e service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:02.507831 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba92f9-1e8d-4d5a-bdcf-31b228b16f3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:02.509487 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-a7ba92f9-1e8d-4d5a-bdcf-31b228b16f3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:02.621972 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba92f9-1e8d-4d5a-bdcf-31b228b16f3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:38:02.623508 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba92f9-1e8d-4d5a-bdcf-31b228b16f3e service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:38:02.644278 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 712/1443] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:38:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:38:04.751216 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ec72c-2e00-4d8b-a0df-de245f5ec7b3 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:04.751887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ec72c-2e00-4d8b-a0df-de245f5ec7b3 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:04.751887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ec72c-2e00-4d8b-a0df-de245f5ec7b3 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:04.753130 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 732/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 05:38:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:38:04.758699 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f68c8-992b-460b-91c8-399304530f40 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:04.759120 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f68c8-992b-460b-91c8-399304530f40 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:04.759251 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f68c8-992b-460b-91c8-399304530f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:04.825354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-785f68c8-992b-460b-91c8-399304530f40 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:38:04.826581 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-785f68c8-992b-460b-91c8-399304530f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:04.839939 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:04.926854 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 713/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 05:38:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:38:05.229015 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ec72c-2e00-4d8b-a0df-de245f5ec7b3 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:38:05.230556 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-383ec72c-2e00-4d8b-a0df-de245f5ec7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:05.332217 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ec72c-2e00-4d8b-a0df-de245f5ec7b3 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:38:05.345092 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:05.412729 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93576a90-3fb1-4a6c-bd21-458f248716f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:38:05.413342 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93576a90-3fb1-4a6c-bd21-458f248716f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:05.413624 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93576a90-3fb1-4a6c-bd21-458f248716f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:05.413900 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93576a90-3fb1-4a6c-bd21-458f248716f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:05.415510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-93576a90-3fb1-4a6c-bd21-458f248716f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:05.526601 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93576a90-3fb1-4a6c-bd21-458f248716f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:38:05.527984 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93576a90-3fb1-4a6c-bd21-458f248716f3 service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:38:05.547184 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 733/1446] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:38:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:38:07.886873 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cf88a-4b5d-47c9-8bf7-fa2e11a4b32d None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:07.887402 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cf88a-4b5d-47c9-8bf7-fa2e11a4b32d None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:07.887402 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cf88a-4b5d-47c9-8bf7-fa2e11a4b32d None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:07.888779 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 714/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 05:38:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:38:07.894391 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d1a637-f705-4951-afda-ac6c2f96501f None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:07.894873 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d1a637-f705-4951-afda-ac6c2f96501f None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:07.894972 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d1a637-f705-4951-afda-ac6c2f96501f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:07.944703 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:07.962119 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-06d1a637-f705-4951-afda-ac6c2f96501f None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:38:07.971289 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-06d1a637-f705-4951-afda-ac6c2f96501f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:08.059371 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 734/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 05:38:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:38:08.344536 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85cf88a-4b5d-47c9-8bf7-fa2e11a4b32d None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:38:08.346117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-b85cf88a-4b5d-47c9-8bf7-fa2e11a4b32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:08.448770 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85cf88a-4b5d-47c9-8bf7-fa2e11a4b32d None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:38:08.488406 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:08.522137 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43c1cd1-d676-427f-93c5-c35143a0a5a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:38:08.522756 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43c1cd1-d676-427f-93c5-c35143a0a5a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:08.523041 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43c1cd1-d676-427f-93c5-c35143a0a5a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:08.523303 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43c1cd1-d676-427f-93c5-c35143a0a5a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:08.524954 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-f43c1cd1-d676-427f-93c5-c35143a0a5a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:08.637203 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43c1cd1-d676-427f-93c5-c35143a0a5a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:38:08.638662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43c1cd1-d676-427f-93c5-c35143a0a5a8 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:38:08.659018 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 715/1449] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:38:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:38:10.709875 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af69ee7-f972-4cd2-9267-f9b62a0ea3fc None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:10.710409 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af69ee7-f972-4cd2-9267-f9b62a0ea3fc None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:10.710409 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af69ee7-f972-4cd2-9267-f9b62a0ea3fc None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:10.711777 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 735/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 05:38:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:38:10.717077 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d38c13-0319-4300-af2a-b092dd3a6b40 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:10.718656 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d38c13-0319-4300-af2a-b092dd3a6b40 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:10.718656 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d38c13-0319-4300-af2a-b092dd3a6b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:10.794030 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-83d38c13-0319-4300-af2a-b092dd3a6b40 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:38:10.795777 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-83d38c13-0319-4300-af2a-b092dd3a6b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:10.818656 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:10.897443 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 716/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 05:38:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:38:11.200963 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af69ee7-f972-4cd2-9267-f9b62a0ea3fc None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:38:11.203611 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-5af69ee7-f972-4cd2-9267-f9b62a0ea3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:11.307092 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af69ee7-f972-4cd2-9267-f9b62a0ea3fc None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:38:11.327581 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:11.384193 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b10cbd7-8401-4c57-8569-5f18354d81b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:38:11.384840 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b10cbd7-8401-4c57-8569-5f18354d81b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:11.385123 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b10cbd7-8401-4c57-8569-5f18354d81b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:11.385405 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b10cbd7-8401-4c57-8569-5f18354d81b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:11.387189 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-0b10cbd7-8401-4c57-8569-5f18354d81b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:11.504098 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b10cbd7-8401-4c57-8569-5f18354d81b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:38:11.505505 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b10cbd7-8401-4c57-8569-5f18354d81b0 service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:38:11.523903 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 736/1452] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:38:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:38:13.511652 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57cceb-fd3b-4890-8997-169c64b3daa9 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:13.512178 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57cceb-fd3b-4890-8997-169c64b3daa9 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:13.512178 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57cceb-fd3b-4890-8997-169c64b3daa9 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:13.513529 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 717/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 05:38:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:38:13.518933 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1bf38-666a-44fc-9536-753fb231a183 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:13.519346 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1bf38-666a-44fc-9536-753fb231a183 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:13.519476 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1bf38-666a-44fc-9536-753fb231a183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:13.583261 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:13.585628 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-18f1bf38-666a-44fc-9536-753fb231a183 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:38:13.586844 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-18f1bf38-666a-44fc-9536-753fb231a183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:13.684212 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 737/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 05:38:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:38:13.972339 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea57cceb-fd3b-4890-8997-169c64b3daa9 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:38:13.973895 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-ea57cceb-fd3b-4890-8997-169c64b3daa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:14.077598 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea57cceb-fd3b-4890-8997-169c64b3daa9 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:38:14.127582 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:14.152046 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9024bad9-7838-47d6-8e11-0a7d8d3cc4d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:38:14.152680 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024bad9-7838-47d6-8e11-0a7d8d3cc4d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:14.152963 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024bad9-7838-47d6-8e11-0a7d8d3cc4d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:14.153241 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024bad9-7838-47d6-8e11-0a7d8d3cc4d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:14.154955 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-9024bad9-7838-47d6-8e11-0a7d8d3cc4d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:14.266503 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9024bad9-7838-47d6-8e11-0a7d8d3cc4d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:38:14.267886 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9024bad9-7838-47d6-8e11-0a7d8d3cc4d8 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:38:14.286459 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 718/1455] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:38:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:38:16.347731 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41afdfb9-1acc-4973-9544-0806b9d6d8c5 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:16.348259 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41afdfb9-1acc-4973-9544-0806b9d6d8c5 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:16.348377 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41afdfb9-1acc-4973-9544-0806b9d6d8c5 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:16.349664 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 738/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 05:38:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:38:16.355256 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b39c8-f091-4042-95b9-6e8236450054 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:16.355655 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b39c8-f091-4042-95b9-6e8236450054 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:16.355804 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b39c8-f091-4042-95b9-6e8236450054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:16.421460 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-9a4b39c8-f091-4042-95b9-6e8236450054 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:38:16.423315 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-9a4b39c8-f091-4042-95b9-6e8236450054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:16.452383 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:16.522162 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 719/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 05:38:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:38:16.807924 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41afdfb9-1acc-4973-9544-0806b9d6d8c5 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:38:16.809530 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-41afdfb9-1acc-4973-9544-0806b9d6d8c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:16.914239 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41afdfb9-1acc-4973-9544-0806b9d6d8c5 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:38:16.942988 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:16.988940 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e653f046-498e-4611-a383-34b1bea1e2fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:38:16.989744 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e653f046-498e-4611-a383-34b1bea1e2fa service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:16.990198 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e653f046-498e-4611-a383-34b1bea1e2fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:16.990935 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e653f046-498e-4611-a383-34b1bea1e2fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:16.992627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-e653f046-498e-4611-a383-34b1bea1e2fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:17.104503 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e653f046-498e-4611-a383-34b1bea1e2fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:38:17.105901 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e653f046-498e-4611-a383-34b1bea1e2fa service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:38:17.125457 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 739/1458] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:38:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:38:19.107406 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e7300-1a47-49ea-a052-dfb1f2ec8ede None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:19.108184 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e7300-1a47-49ea-a052-dfb1f2ec8ede None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:19.108184 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e7300-1a47-49ea-a052-dfb1f2ec8ede None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:19.109724 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 720/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 05:38:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:38:19.115853 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f56b979-fdb9-4e31-9c13-38be02b03545 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:19.115853 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f56b979-fdb9-4e31-9c13-38be02b03545 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:19.115853 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f56b979-fdb9-4e31-9c13-38be02b03545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:19.182597 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-7f56b979-fdb9-4e31-9c13-38be02b03545 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:38:19.183818 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-7f56b979-fdb9-4e31-9c13-38be02b03545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:19.193222 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:19.283838 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 740/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 05:38:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:38:19.568939 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5e7300-1a47-49ea-a052-dfb1f2ec8ede None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:38:19.570482 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-ac5e7300-1a47-49ea-a052-dfb1f2ec8ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:19.678352 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5e7300-1a47-49ea-a052-dfb1f2ec8ede None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:38:19.732898 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:19.754579 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e711e0c-8028-4094-94db-37b66dc1c233 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:38:19.755185 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e711e0c-8028-4094-94db-37b66dc1c233 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:19.755467 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e711e0c-8028-4094-94db-37b66dc1c233 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:19.755738 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e711e0c-8028-4094-94db-37b66dc1c233 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:19.757440 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-0e711e0c-8028-4094-94db-37b66dc1c233 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:19.880505 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e711e0c-8028-4094-94db-37b66dc1c233 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:38:19.882076 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e711e0c-8028-4094-94db-37b66dc1c233 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:38:19.901902 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 721/1461] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:38:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:38:21.882329 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f476e04-04b9-45c7-bca6-a9dcc6106d75 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:21.882901 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f476e04-04b9-45c7-bca6-a9dcc6106d75 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:21.882901 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f476e04-04b9-45c7-bca6-a9dcc6106d75 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:21.884242 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 741/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 05:38:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:38:21.890568 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64688298-7ad3-4b08-94a9-7fce42bec36b None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:21.891113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64688298-7ad3-4b08-94a9-7fce42bec36b None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:21.891220 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64688298-7ad3-4b08-94a9-7fce42bec36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:21.962248 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-64688298-7ad3-4b08-94a9-7fce42bec36b None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:38:21.963486 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-64688298-7ad3-4b08-94a9-7fce42bec36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:22.004817 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:22.064823 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 722/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 05:38:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:38:22.348991 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f476e04-04b9-45c7-bca6-a9dcc6106d75 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:38:22.350538 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-8f476e04-04b9-45c7-bca6-a9dcc6106d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:22.457805 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f476e04-04b9-45c7-bca6-a9dcc6106d75 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:38:22.496580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:22.538354 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae71c195-a438-4747-8205-002d294baf5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:38:22.539010 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae71c195-a438-4747-8205-002d294baf5a service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:22.539292 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae71c195-a438-4747-8205-002d294baf5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:22.539571 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae71c195-a438-4747-8205-002d294baf5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:22.541269 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-ae71c195-a438-4747-8205-002d294baf5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:22.653062 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae71c195-a438-4747-8205-002d294baf5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:38:22.654511 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae71c195-a438-4747-8205-002d294baf5a service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:38:22.675105 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 742/1464] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:38:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:38:24.639788 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db097d19-1b2c-44a0-bb06-643b2b691315 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:24.640483 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db097d19-1b2c-44a0-bb06-643b2b691315 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:24.640483 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db097d19-1b2c-44a0-bb06-643b2b691315 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:24.642287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 723/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 05:38:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:38:24.648015 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d9fce-2e9b-4759-9aa0-f66db23f5e8f None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:24.648514 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d9fce-2e9b-4759-9aa0-f66db23f5e8f None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:24.648618 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d9fce-2e9b-4759-9aa0-f66db23f5e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:24.716706 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-fc9d9fce-2e9b-4759-9aa0-f66db23f5e8f None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:38:24.717850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-fc9d9fce-2e9b-4759-9aa0-f66db23f5e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:24.737442 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:24.817961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 743/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 05:38:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:38:25.110354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db097d19-1b2c-44a0-bb06-643b2b691315 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:38:25.111993 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-db097d19-1b2c-44a0-bb06-643b2b691315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:25.227077 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db097d19-1b2c-44a0-bb06-643b2b691315 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:38:25.296374 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:25.303326 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b42acd-0a8e-401c-ae26-c5f9684f6beb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:38:25.303974 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b42acd-0a8e-401c-ae26-c5f9684f6beb service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:25.304257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b42acd-0a8e-401c-ae26-c5f9684f6beb service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:25.304566 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b42acd-0a8e-401c-ae26-c5f9684f6beb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:25.308151 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-23b42acd-0a8e-401c-ae26-c5f9684f6beb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:25.416381 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b42acd-0a8e-401c-ae26-c5f9684f6beb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:38:25.417890 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b42acd-0a8e-401c-ae26-c5f9684f6beb service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:38:25.437601 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 724/1467] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:38:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:38:27.424681 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae6a8f-7e7f-4341-8c7e-553815cffb84 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:27.425314 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae6a8f-7e7f-4341-8c7e-553815cffb84 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:27.425314 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae6a8f-7e7f-4341-8c7e-553815cffb84 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:27.426684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 744/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 05:38:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:38:27.432256 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6eda7-ecb6-47b3-83c6-0aa1a21d65f6 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:27.432785 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6eda7-ecb6-47b3-83c6-0aa1a21d65f6 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:27.432956 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6eda7-ecb6-47b3-83c6-0aa1a21d65f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:27.502973 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-f9c6eda7-ecb6-47b3-83c6-0aa1a21d65f6 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:38:27.504095 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-f9c6eda7-ecb6-47b3-83c6-0aa1a21d65f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:27.569651 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:27.620282 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 725/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 05:38:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:38:27.903848 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae6a8f-7e7f-4341-8c7e-553815cffb84 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:38:27.905452 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-83ae6a8f-7e7f-4341-8c7e-553815cffb84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:28.014595 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae6a8f-7e7f-4341-8c7e-553815cffb84 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:38:28.063223 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:28.089923 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d4d28c-f0ae-4147-871e-c0f996ebe5e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:38:28.090536 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d4d28c-f0ae-4147-871e-c0f996ebe5e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:28.090816 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d4d28c-f0ae-4147-871e-c0f996ebe5e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:28.091089 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d4d28c-f0ae-4147-871e-c0f996ebe5e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:28.092721 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-13d4d28c-f0ae-4147-871e-c0f996ebe5e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:28.210638 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d4d28c-f0ae-4147-871e-c0f996ebe5e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:38:28.212083 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d4d28c-f0ae-4147-871e-c0f996ebe5e1 service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:38:28.230552 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 745/1470] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:38:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:38:37.618520 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=22 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:38.228234 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=66 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:48.308145 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816ef5ee-31a4-48e1-8e85-d57df03ce215 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:48.308756 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816ef5ee-31a4-48e1-8e85-d57df03ce215 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:48.308756 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816ef5ee-31a4-48e1-8e85-d57df03ce215 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:48.310107 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 726/1471] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:38:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:38:48.316013 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b8ac2-1ebd-4355-92a6-b851cb6e569b None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:48.316541 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b8ac2-1ebd-4355-92a6-b851cb6e569b None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:48.316677 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b8ac2-1ebd-4355-92a6-b851cb6e569b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:48.382443 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-e03b8ac2-1ebd-4355-92a6-b851cb6e569b None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:38:48.383663 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-e03b8ac2-1ebd-4355-92a6-b851cb6e569b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:48.490545 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 746/1472] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:38:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:38:48.789822 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a0cfc-570e-4313-bdcf-21bc838c1297 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:48.790106 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a0cfc-570e-4313-bdcf-21bc838c1297 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:48.790385 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a0cfc-570e-4313-bdcf-21bc838c1297 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:48.791762 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 727/1473] 10.222.0.15 () {54 vars in 967 bytes} [Wed Aug 24 05:38:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:38:48.797791 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4cdeb7-d70f-4fb5-a77f-e376c7a522ef None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:48.798118 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4cdeb7-d70f-4fb5-a77f-e376c7a522ef None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:48.798341 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4cdeb7-d70f-4fb5-a77f-e376c7a522ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:48.882771 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-4c4cdeb7-d70f-4fb5-a77f-e376c7a522ef None None] MFA Rules not processed for user `9aee9c524a974a248c3bf363444f584a`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:38:48.884002 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-4c4cdeb7-d70f-4fb5-a77f-e376c7a522ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:48.906429 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:48.964943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 747/1474] 10.222.0.15 () {58 vars in 1064 bytes} [Wed Aug 24 05:38:48 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:38:48.973401 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a0cfc-570e-4313-bdcf-21bc838c1297 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:38:48.974971 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-0c7a0cfc-570e-4313-bdcf-21bc838c1297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:49.083102 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a0cfc-570e-4313-bdcf-21bc838c1297 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:38:49.161536 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844bd37d-ec82-401b-b8e5-bc33c04d0e95 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9aee9c524a974a248c3bf363444f584a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:38:49.162136 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844bd37d-ec82-401b-b8e5-bc33c04d0e95 service glance] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:49.162413 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844bd37d-ec82-401b-b8e5-bc33c04d0e95 service glance] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:49.162947 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844bd37d-ec82-401b-b8e5-bc33c04d0e95 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:49.164575 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-844bd37d-ec82-401b-b8e5-bc33c04d0e95 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:49.221220 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:49.278422 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844bd37d-ec82-401b-b8e5-bc33c04d0e95 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:38:49.280068 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844bd37d-ec82-401b-b8e5-bc33c04d0e95 service glance] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:38:49.301775 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 728/1475] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:38:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:38:51.356533 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9705f-3ce6-43a7-8067-34d215745263 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:51.357083 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9705f-3ce6-43a7-8067-34d215745263 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:51.357083 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9705f-3ce6-43a7-8067-34d215745263 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:51.359732 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 748/1476] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:38:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:38:51.365918 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a28b0-de5f-464f-8232-1a37de66a83a None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:51.366440 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a28b0-de5f-464f-8232-1a37de66a83a None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:51.366588 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a28b0-de5f-464f-8232-1a37de66a83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:51.458903 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-f37a28b0-de5f-464f-8232-1a37de66a83a None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:38:51.460066 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-f37a28b0-de5f-464f-8232-1a37de66a83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:51.573924 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 729/1477] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:38:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:38:51.881264 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d9705f-3ce6-43a7-8067-34d215745263 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:38:51.882797 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-02d9705f-3ce6-43a7-8067-34d215745263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:52.053286 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d9705f-3ce6-43a7-8067-34d215745263 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:38:52.123166 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:52.194153 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dce068-ea05-4f08-b1ea-51dc81c57819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:38:52.196089 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dce068-ea05-4f08-b1ea-51dc81c57819 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:52.196370 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dce068-ea05-4f08-b1ea-51dc81c57819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:52.196707 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dce068-ea05-4f08-b1ea-51dc81c57819 admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:52.197364 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dce068-ea05-4f08-b1ea-51dc81c57819 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:38:52.203426 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: WARNING py.warnings [None req-e7dce068-ea05-4f08-b1ea-51dc81c57819 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 05:38:52.203426 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: warnings.warn(msg) Aug 24 05:38:52.203426 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: Aug 24 05:38:52.204619 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dce068-ea05-4f08-b1ea-51dc81c57819 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:38:52.228633 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 749/1478] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Aug 24 05:38:51 2022] GET /identity/v3/projects => generated 2553 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:38:54.182265 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed351fe9-fbfb-401f-a6d9-f43c1697fffc None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:54.182815 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed351fe9-fbfb-401f-a6d9-f43c1697fffc None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:54.182815 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed351fe9-fbfb-401f-a6d9-f43c1697fffc None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:54.184165 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 730/1479] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:38:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:38:54.190603 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad277f9d-d0d0-4b25-8555-1728673bc117 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:54.191272 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad277f9d-d0d0-4b25-8555-1728673bc117 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:54.191272 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad277f9d-d0d0-4b25-8555-1728673bc117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:54.265461 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-ad277f9d-d0d0-4b25-8555-1728673bc117 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:38:54.266699 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-ad277f9d-d0d0-4b25-8555-1728673bc117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:54.367172 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 750/1480] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:38:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:38:54.668148 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed351fe9-fbfb-401f-a6d9-f43c1697fffc None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:38:54.669742 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-ed351fe9-fbfb-401f-a6d9-f43c1697fffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:54.680375 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:54.820985 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed351fe9-fbfb-401f-a6d9-f43c1697fffc None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:38:54.903616 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:54.938481 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb339fa-d7e1-4f93-a508-b1a370e9271a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:38:54.939089 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb339fa-d7e1-4f93-a508-b1a370e9271a service nova] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:54.939511 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb339fa-d7e1-4f93-a508-b1a370e9271a service nova] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:54.939794 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb339fa-d7e1-4f93-a508-b1a370e9271a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:54.941544 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-bbb339fa-d7e1-4f93-a508-b1a370e9271a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:55.031450 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb339fa-d7e1-4f93-a508-b1a370e9271a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:38:55.033126 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb339fa-d7e1-4f93-a508-b1a370e9271a service nova] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:38:55.052064 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 731/1481] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:38:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:38:57.222983 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b524f68-e857-459c-bd97-06d3d91d7b5b None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:57.223568 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b524f68-e857-459c-bd97-06d3d91d7b5b None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:57.223697 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b524f68-e857-459c-bd97-06d3d91d7b5b None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:57.225793 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 751/1482] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:38:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:38:57.233588 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f1c1f0-beb7-4584-b931-d63e1d865fc2 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:57.234211 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f1c1f0-beb7-4584-b931-d63e1d865fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:57.234211 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f1c1f0-beb7-4584-b931-d63e1d865fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:57.304686 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-45f1c1f0-beb7-4584-b931-d63e1d865fc2 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:38:57.305909 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-45f1c1f0-beb7-4584-b931-d63e1d865fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:57.356254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:57.422835 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 732/1483] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:38:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:38:57.735176 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b524f68-e857-459c-bd97-06d3d91d7b5b None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:38:57.736775 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-4b524f68-e857-459c-bd97-06d3d91d7b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:57.760120 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:57.846309 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b524f68-e857-459c-bd97-06d3d91d7b5b None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:38:57.919620 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61fbce5-90b2-48bc-b029-16ae1512e396 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:38:57.920507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61fbce5-90b2-48bc-b029-16ae1512e396 service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:57.920805 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61fbce5-90b2-48bc-b029-16ae1512e396 service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:57.931086 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61fbce5-90b2-48bc-b029-16ae1512e396 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:57.932986 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-a61fbce5-90b2-48bc-b029-16ae1512e396 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:58.013572 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:58.051195 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61fbce5-90b2-48bc-b029-16ae1512e396 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:38:58.052677 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61fbce5-90b2-48bc-b029-16ae1512e396 service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:38:58.071389 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 752/1484] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:38:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:39:00.071969 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da923e9-da81-4a08-a368-4defc38f31d2 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:00.072590 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da923e9-da81-4a08-a368-4defc38f31d2 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:00.072590 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da923e9-da81-4a08-a368-4defc38f31d2 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:00.074065 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 733/1485] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:39:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:39:00.081408 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5efea2-5bdf-4a86-9b6d-85c1c4a781e0 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:00.081919 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5efea2-5bdf-4a86-9b6d-85c1c4a781e0 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:00.082060 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5efea2-5bdf-4a86-9b6d-85c1c4a781e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:00.151598 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-4a5efea2-5bdf-4a86-9b6d-85c1c4a781e0 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:39:00.152905 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-4a5efea2-5bdf-4a86-9b6d-85c1c4a781e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:00.262111 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 753/1486] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:39:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:39:00.559277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da923e9-da81-4a08-a368-4defc38f31d2 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:39:00.561085 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-4da923e9-da81-4a08-a368-4defc38f31d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:00.666684 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da923e9-da81-4a08-a368-4defc38f31d2 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:39:00.740009 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:00.741923 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab9636f-a78c-4336-95d7-69ba839d8bb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:39:00.742534 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9636f-a78c-4336-95d7-69ba839d8bb8 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:00.742888 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9636f-a78c-4336-95d7-69ba839d8bb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:00.743170 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9636f-a78c-4336-95d7-69ba839d8bb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:00.744844 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-8ab9636f-a78c-4336-95d7-69ba839d8bb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:00.864634 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab9636f-a78c-4336-95d7-69ba839d8bb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:39:00.865753 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab9636f-a78c-4336-95d7-69ba839d8bb8 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:39:00.885199 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 734/1487] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:39:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:39:02.918515 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3c84d-6821-4508-b581-313f4841e594 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:02.919092 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3c84d-6821-4508-b581-313f4841e594 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:02.919092 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3c84d-6821-4508-b581-313f4841e594 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:02.921148 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 754/1488] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:39:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:39:02.926763 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e505f0c-b44a-48bc-89c5-f65e18e62945 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:02.927225 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e505f0c-b44a-48bc-89c5-f65e18e62945 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:02.927342 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e505f0c-b44a-48bc-89c5-f65e18e62945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:02.993675 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-9e505f0c-b44a-48bc-89c5-f65e18e62945 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:39:02.994912 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-9e505f0c-b44a-48bc-89c5-f65e18e62945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:03.051685 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:03.097638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 735/1489] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:39:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:39:03.389518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb3c84d-6821-4508-b581-313f4841e594 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:39:03.391081 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-3bb3c84d-6821-4508-b581-313f4841e594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:03.432647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:03.502313 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb3c84d-6821-4508-b581-313f4841e594 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:39:03.578298 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e877052-372b-4a6e-a859-fde3e212ee90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:39:03.578907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e877052-372b-4a6e-a859-fde3e212ee90 service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:03.579190 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e877052-372b-4a6e-a859-fde3e212ee90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:03.579475 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e877052-372b-4a6e-a859-fde3e212ee90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:03.581114 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-3e877052-372b-4a6e-a859-fde3e212ee90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:03.666938 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:03.697791 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e877052-372b-4a6e-a859-fde3e212ee90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:39:03.699250 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e877052-372b-4a6e-a859-fde3e212ee90 service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:39:03.719980 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 755/1490] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:39:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:39:06.203147 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8365ad-a4f3-4b30-974b-e63d09946202 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:06.203685 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8365ad-a4f3-4b30-974b-e63d09946202 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:06.203685 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8365ad-a4f3-4b30-974b-e63d09946202 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:06.205122 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 736/1491] 10.222.0.15 () {54 vars in 927 bytes} [Wed Aug 24 05:39:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:39:06.210545 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e4f16-44a5-432a-853f-3d1d35b03ec7 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:06.210993 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e4f16-44a5-432a-853f-3d1d35b03ec7 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:06.211176 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e4f16-44a5-432a-853f-3d1d35b03ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:06.278183 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-ad8e4f16-44a5-432a-853f-3d1d35b03ec7 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:39:06.279998 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-ad8e4f16-44a5-432a-853f-3d1d35b03ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:06.384320 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 756/1492] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Aug 24 05:39:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:39:06.686328 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8365ad-a4f3-4b30-974b-e63d09946202 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:39:06.688092 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-bc8365ad-a4f3-4b30-974b-e63d09946202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:06.808165 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8365ad-a4f3-4b30-974b-e63d09946202 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:39:06.885911 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c48ab1-f6cd-4d7a-b911-7efc41140002 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:39:06.886533 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c48ab1-f6cd-4d7a-b911-7efc41140002 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:06.886815 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c48ab1-f6cd-4d7a-b911-7efc41140002 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:06.887090 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c48ab1-f6cd-4d7a-b911-7efc41140002 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:06.889582 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-45c48ab1-f6cd-4d7a-b911-7efc41140002 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:06.900267 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:07.009678 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c48ab1-f6cd-4d7a-b911-7efc41140002 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:39:07.011122 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c48ab1-f6cd-4d7a-b911-7efc41140002 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:39:07.031740 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 737/1493] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:39:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:39:16.382517 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=82 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:17.028507 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=50 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:38.604960 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974550fb-7169-4fa9-9b65-1a2d6e987ea4 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:38.605719 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974550fb-7169-4fa9-9b65-1a2d6e987ea4 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:38.605719 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974550fb-7169-4fa9-9b65-1a2d6e987ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:38.693401 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-974550fb-7169-4fa9-9b65-1a2d6e987ea4 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:40:38.694667 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-974550fb-7169-4fa9-9b65-1a2d6e987ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:38.829946 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 757/1494] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:40:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:40:38.839100 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c128a1d-eae7-46a1-a191-f3e5b2c2203e None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:38.839950 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c128a1d-eae7-46a1-a191-f3e5b2c2203e None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:38.839950 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c128a1d-eae7-46a1-a191-f3e5b2c2203e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:38.957701 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-6c128a1d-eae7-46a1-a191-f3e5b2c2203e None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:40:38.958926 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-6c128a1d-eae7-46a1-a191-f3e5b2c2203e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:39.092370 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 738/1495] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:40:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:40:39.097094 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974550fb-7169-4fa9-9b65-1a2d6e987ea4 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:40:39.098361 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-974550fb-7169-4fa9-9b65-1a2d6e987ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:39.111333 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974550fb-7169-4fa9-9b65-1a2d6e987ea4 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:40:39.190356 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:39.233293 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65721d69-43e7-463e-ad5c-dce981877a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:40:39.233835 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65721d69-43e7-463e-ad5c-dce981877a2d admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:39.234120 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65721d69-43e7-463e-ad5c-dce981877a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:39.234401 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65721d69-43e7-463e-ad5c-dce981877a2d admin admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:39.235101 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65721d69-43e7-463e-ad5c-dce981877a2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:40:39.235411 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65721d69-43e7-463e-ad5c-dce981877a2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:40:39.236520 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: WARNING py.warnings [None req-65721d69-43e7-463e-ad5c-dce981877a2d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 05:40:39.236520 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: warnings.warn(msg) Aug 24 05:40:39.236520 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: Aug 24 05:40:39.237458 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65721d69-43e7-463e-ad5c-dce981877a2d admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:40:39.252473 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 758/1496] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:40:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:40:39.262844 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c128a1d-eae7-46a1-a191-f3e5b2c2203e None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:40:39.272944 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c128a1d-eae7-46a1-a191-f3e5b2c2203e None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:40:39.345943 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:39.395354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c61ed3a-3674-4994-a7fb-71c4b4febf21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:40:39.395966 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61ed3a-3674-4994-a7fb-71c4b4febf21 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:39.396244 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61ed3a-3674-4994-a7fb-71c4b4febf21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:39.396559 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61ed3a-3674-4994-a7fb-71c4b4febf21 admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:39.397201 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c61ed3a-3674-4994-a7fb-71c4b4febf21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:40:39.399044 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: WARNING py.warnings [None req-7c61ed3a-3674-4994-a7fb-71c4b4febf21 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 05:40:39.399044 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: warnings.warn(msg) Aug 24 05:40:39.399044 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: Aug 24 05:40:39.401243 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c61ed3a-3674-4994-a7fb-71c4b4febf21 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:40:39.575541 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 739/1497] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:40:39 2022] POST /identity/v3/projects => generated 377 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:40:39.580369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65721d69-43e7-463e-ad5c-dce981877a2d admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:40:39.582000 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-65721d69-43e7-463e-ad5c-dce981877a2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:39.619525 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65721d69-43e7-463e-ad5c-dce981877a2d admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:40:39.728260 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd054fd0-8538-41e0-bcce-981db72430c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:40:39.729539 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd054fd0-8538-41e0-bcce-981db72430c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:39.730061 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd054fd0-8538-41e0-bcce-981db72430c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:39.730414 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd054fd0-8538-41e0-bcce-981db72430c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:39.731119 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd054fd0-8538-41e0-bcce-981db72430c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:40:39.733615 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: WARNING py.warnings [None req-dd054fd0-8538-41e0-bcce-981db72430c4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 05:40:39.733615 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: warnings.warn(msg) Aug 24 05:40:39.733615 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: Aug 24 05:40:39.735351 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd054fd0-8538-41e0-bcce-981db72430c4 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:40:39.866200 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 759/1498] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:40:39 2022] POST /identity/v3/users => generated 289 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:40:39.871327 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c61ed3a-3674-4994-a7fb-71c4b4febf21 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:40:39.873179 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-7c61ed3a-3674-4994-a7fb-71c4b4febf21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:39.951463 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c61ed3a-3674-4994-a7fb-71c4b4febf21 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:40:40.046897 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:40.058690 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b1d6b-e2ac-4fdc-a459-4a2b62c989e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:40:40.062275 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b1d6b-e2ac-4fdc-a459-4a2b62c989e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:40.062275 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b1d6b-e2ac-4fdc-a459-4a2b62c989e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:40.062275 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b1d6b-e2ac-4fdc-a459-4a2b62c989e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:40.062275 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9b1d6b-e2ac-4fdc-a459-4a2b62c989e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:40:40.062275 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: WARNING py.warnings [None req-be9b1d6b-e2ac-4fdc-a459-4a2b62c989e1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 05:40:40.062275 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: warnings.warn(msg) Aug 24 05:40:40.062275 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: Aug 24 05:40:40.063167 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9b1d6b-e2ac-4fdc-a459-4a2b62c989e1 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:40:40.076554 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 740/1499] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:40:39 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:40:40.081130 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd054fd0-8538-41e0-bcce-981db72430c4 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:40:40.082724 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-dd054fd0-8538-41e0-bcce-981db72430c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:40.095370 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd054fd0-8538-41e0-bcce-981db72430c4 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:40:40.125801 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:40.212355 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4ba68e-6e64-4997-a8a9-2caa3b48b0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:40:40.213095 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ba68e-6e64-4997-a8a9-2caa3b48b0ae admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:40.213372 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ba68e-6e64-4997-a8a9-2caa3b48b0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:40.213650 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ba68e-6e64-4997-a8a9-2caa3b48b0ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:40.214298 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4ba68e-6e64-4997-a8a9-2caa3b48b0ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:40:40.215641 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: WARNING py.warnings [None req-ef4ba68e-6e64-4997-a8a9-2caa3b48b0ae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 05:40:40.215641 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: warnings.warn(msg) Aug 24 05:40:40.215641 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: Aug 24 05:40:40.216571 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4ba68e-6e64-4997-a8a9-2caa3b48b0ae admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:40:40.231187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 760/1500] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:40:40 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:40:40.235628 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b1d6b-e2ac-4fdc-a459-4a2b62c989e1 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:40:40.237306 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-be9b1d6b-e2ac-4fdc-a459-4a2b62c989e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:40.272327 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b1d6b-e2ac-4fdc-a459-4a2b62c989e1 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:40:40.391165 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e5648b-8f23-487c-98c2-6b170b990a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:40:40.391761 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5648b-8f23-487c-98c2-6b170b990a0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:40.392041 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5648b-8f23-487c-98c2-6b170b990a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:40.393108 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5648b-8f23-487c-98c2-6b170b990a0c admin admin] PATH_INFO: `/v3/projects/b57ffe1f208e48b29a10ef7d278a5d60/users/e9bbc3fc24db4e32bffb028ff46c3e81/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:40.422525 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e5648b-8f23-487c-98c2-6b170b990a0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b57ffe1f208e48b29a10ef7d278a5d60, user_id=e9bbc3fc24db4e32bffb028ff46c3e81, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:40:40.423596 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: WARNING py.warnings [None req-77e5648b-8f23-487c-98c2-6b170b990a0c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 05:40:40.423596 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: warnings.warn(msg) Aug 24 05:40:40.423596 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: Aug 24 05:40:40.425668 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e5648b-8f23-487c-98c2-6b170b990a0c admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:40:40.546881 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 741/1501] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:40:40 2022] PUT /identity/v3/projects/b57ffe1f208e48b29a10ef7d278a5d60/users/e9bbc3fc24db4e32bffb028ff46c3e81/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:40:40.560846 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2798aa-b93b-418d-948d-b7c33a321e39 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:40.561272 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2798aa-b93b-418d-948d-b7c33a321e39 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:40.561636 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2798aa-b93b-418d-948d-b7c33a321e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:40.631753 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-dd2798aa-b93b-418d-948d-b7c33a321e39 None None] MFA Rules not processed for user `e9bbc3fc24db4e32bffb028ff46c3e81`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:40:40.633015 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-dd2798aa-b93b-418d-948d-b7c33a321e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:40.735600 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 761/1502] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:40:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:40:40.751172 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3944b5c-374f-4afa-9794-195be379edb5 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:40.751458 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3944b5c-374f-4afa-9794-195be379edb5 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:40.751730 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3944b5c-374f-4afa-9794-195be379edb5 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:40.753312 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 742/1503] 10.222.0.15 () {54 vars in 964 bytes} [Wed Aug 24 05:40:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:40:40.759030 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df78cb3f-84d4-43d2-8938-8e5bd51aee9d None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:40.759366 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df78cb3f-84d4-43d2-8938-8e5bd51aee9d None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:40.759780 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df78cb3f-84d4-43d2-8938-8e5bd51aee9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:40.802230 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:40.860227 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-df78cb3f-84d4-43d2-8938-8e5bd51aee9d None None] MFA Rules not processed for user `47bfd618ef39498f91fe7731cd40253b`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:40:40.861641 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-df78cb3f-84d4-43d2-8938-8e5bd51aee9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:40.957909 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 762/1504] 10.222.0.15 () {58 vars in 1061 bytes} [Wed Aug 24 05:40:40 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:40:40.966507 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3944b5c-374f-4afa-9794-195be379edb5 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:40:40.968301 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-c3944b5c-374f-4afa-9794-195be379edb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:41.034839 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:41.111788 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3944b5c-374f-4afa-9794-195be379edb5 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:40:41.209352 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1e3923-fcbc-452d-8c20-7ad384ae9e5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:40:41.210310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1e3923-fcbc-452d-8c20-7ad384ae9e5a service cinder] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:41.210806 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1e3923-fcbc-452d-8c20-7ad384ae9e5a service cinder] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:41.211305 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1e3923-fcbc-452d-8c20-7ad384ae9e5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:41.214730 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-1a1e3923-fcbc-452d-8c20-7ad384ae9e5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:41.342549 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1e3923-fcbc-452d-8c20-7ad384ae9e5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:40:41.344059 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1e3923-fcbc-452d-8c20-7ad384ae9e5a service cinder] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:40:41.348042 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:41.366779 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 743/1505] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:40:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:40:41.421456 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26562927-1628-4da2-8698-6c18b16a849e None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:41.421825 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26562927-1628-4da2-8698-6c18b16a849e None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:41.422239 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26562927-1628-4da2-8698-6c18b16a849e None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:41.424111 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 763/1506] 10.222.0.15 () {52 vars in 830 bytes} [Wed Aug 24 05:40:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:40:41.428308 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1e3923-fcbc-452d-8c20-7ad384ae9e5a service cinder] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:40:41.429601 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-1a1e3923-fcbc-452d-8c20-7ad384ae9e5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:41.545276 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1e3923-fcbc-452d-8c20-7ad384ae9e5a service cinder] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:40:41.655975 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5397fb-a569-45f8-ab3e-2c180898f5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:40:41.656647 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5397fb-a569-45f8-ab3e-2c180898f5ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:41.656931 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5397fb-a569-45f8-ab3e-2c180898f5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:41.657206 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5397fb-a569-45f8-ab3e-2c180898f5ae admin admin] PATH_INFO: `/v3/users/e9bbc3fc24db4e32bffb028ff46c3e81` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:41.673087 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5397fb-a569-45f8-ab3e-2c180898f5ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9bbc3fc24db4e32bffb028ff46c3e81)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:40:41.674166 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: WARNING py.warnings [None req-3c5397fb-a569-45f8-ab3e-2c180898f5ae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 05:40:41.674166 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: warnings.warn(msg) Aug 24 05:40:41.674166 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: Aug 24 05:40:41.675397 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5397fb-a569-45f8-ab3e-2c180898f5ae admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:40:41.687425 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:41.937928 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-3c5397fb-a569-45f8-ab3e-2c180898f5ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9bbc3fc24db4e32bffb028ff46c3e81'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:40:41.957468 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-3c5397fb-a569-45f8-ab3e-2c180898f5ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9bbc3fc24db4e32bffb028ff46c3e81'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:40:41.957973 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-3c5397fb-a569-45f8-ab3e-2c180898f5ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9bbc3fc24db4e32bffb028ff46c3e81'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:40:42.074063 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-3c5397fb-a569-45f8-ab3e-2c180898f5ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9bbc3fc24db4e32bffb028ff46c3e81'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:40:42.109537 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-3c5397fb-a569-45f8-ab3e-2c180898f5ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9bbc3fc24db4e32bffb028ff46c3e81'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:40:42.116056 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 744/1507] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:40:41 2022] DELETE /identity/v3/users/e9bbc3fc24db4e32bffb028ff46c3e81 => generated 0 bytes in 688 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:40:42.120785 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba271d7-205c-4a3c-a21a-0cffba0178f9 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:42.121458 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba271d7-205c-4a3c-a21a-0cffba0178f9 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:42.121458 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba271d7-205c-4a3c-a21a-0cffba0178f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:42.190987 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-fba271d7-205c-4a3c-a21a-0cffba0178f9 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:40:42.192181 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-fba271d7-205c-4a3c-a21a-0cffba0178f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:42.277868 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:42.291064 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 764/1508] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:40:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:40:42.301019 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5397fb-a569-45f8-ab3e-2c180898f5ae admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:40:42.302478 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-3c5397fb-a569-45f8-ab3e-2c180898f5ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:42.418839 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5397fb-a569-45f8-ab3e-2c180898f5ae admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:40:42.489333 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:42.514280 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bca4133-9b85-41bb-9261-bf718ca13b27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:40:42.514916 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca4133-9b85-41bb-9261-bf718ca13b27 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:42.515215 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca4133-9b85-41bb-9261-bf718ca13b27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:42.515493 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca4133-9b85-41bb-9261-bf718ca13b27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:42.517362 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-3bca4133-9b85-41bb-9261-bf718ca13b27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:42.704931 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bca4133-9b85-41bb-9261-bf718ca13b27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:40:42.707738 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bca4133-9b85-41bb-9261-bf718ca13b27 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:40:42.736803 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 745/1509] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:40:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:40:43.243456 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba271d7-205c-4a3c-a21a-0cffba0178f9 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:40:43.246080 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-fba271d7-205c-4a3c-a21a-0cffba0178f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:43.259101 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba271d7-205c-4a3c-a21a-0cffba0178f9 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:40:43.394092 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1b560-ad87-43a0-a567-730a48ea344e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:40:43.394743 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1b560-ad87-43a0-a567-730a48ea344e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:43.395023 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1b560-ad87-43a0-a567-730a48ea344e admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:43.395293 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1b560-ad87-43a0-a567-730a48ea344e admin admin] PATH_INFO: `/v3/projects/b57ffe1f208e48b29a10ef7d278a5d60` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:43.409570 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f1b560-ad87-43a0-a567-730a48ea344e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b57ffe1f208e48b29a10ef7d278a5d60)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:40:43.410646 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: WARNING py.warnings [None req-d3f1b560-ad87-43a0-a567-730a48ea344e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 05:40:43.410646 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: warnings.warn(msg) Aug 24 05:40:43.410646 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: Aug 24 05:40:43.412073 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f1b560-ad87-43a0-a567-730a48ea344e admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:40:43.588090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-d3f1b560-ad87-43a0-a567-730a48ea344e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b57ffe1f208e48b29a10ef7d278a5d60', 'request_id': 'req-d3f1b560-ad87-43a0-a567-730a48ea344e'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:40:43.615636 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-d3f1b560-ad87-43a0-a567-730a48ea344e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b57ffe1f208e48b29a10ef7d278a5d60', 'request_id': 'req-d3f1b560-ad87-43a0-a567-730a48ea344e'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:40:43.747021 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-d3f1b560-ad87-43a0-a567-730a48ea344e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b57ffe1f208e48b29a10ef7d278a5d60', 'request_id': 'req-d3f1b560-ad87-43a0-a567-730a48ea344e'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:40:43.771683 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-d3f1b560-ad87-43a0-a567-730a48ea344e admin admin] The token cache is being invalidate because project b57ffe1f208e48b29a10ef7d278a5d60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:40:43.771976 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-d3f1b560-ad87-43a0-a567-730a48ea344e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:40:43.772447 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-d3f1b560-ad87-43a0-a567-730a48ea344e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:40:43.773888 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 765/1510] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:40:43 2022] DELETE /identity/v3/projects/b57ffe1f208e48b29a10ef7d278a5d60 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:40:52.850890 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=62 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:53.769421 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:42:20.975888 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b6fe4-6961-4c56-bea4-9cfa2b40480e None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:20.976776 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b6fe4-6961-4c56-bea4-9cfa2b40480e None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:20.976776 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b6fe4-6961-4c56-bea4-9cfa2b40480e None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:20.978374 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 746/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 05:42:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:42:20.983591 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b6014a-f7b9-462a-8fb8-dcb0430e407d None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:20.984094 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b6014a-f7b9-462a-8fb8-dcb0430e407d None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:20.984193 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b6014a-f7b9-462a-8fb8-dcb0430e407d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:21.086492 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-d1b6014a-f7b9-462a-8fb8-dcb0430e407d None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:42:21.087661 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-d1b6014a-f7b9-462a-8fb8-dcb0430e407d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:42:21.233207 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 766/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 05:42:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:42:21.407357 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b6fe4-6961-4c56-bea4-9cfa2b40480e None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:42:21.409690 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-298b6fe4-6961-4c56-bea4-9cfa2b40480e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:42:21.559899 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b6fe4-6961-4c56-bea4-9cfa2b40480e None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:42:21.567601 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78aacb8-c7e0-4fc4-b4c6-0f1a859a8f52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:42:21.568479 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78aacb8-c7e0-4fc4-b4c6-0f1a859a8f52 service nova] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:21.568849 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78aacb8-c7e0-4fc4-b4c6-0f1a859a8f52 service nova] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:21.569131 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78aacb8-c7e0-4fc4-b4c6-0f1a859a8f52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:21.578963 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78aacb8-c7e0-4fc4-b4c6-0f1a859a8f52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:42:21.580643 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78aacb8-c7e0-4fc4-b4c6-0f1a859a8f52 service nova] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:42:21.590811 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 747/1513] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:42:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:42:22.459832 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a81452-0f10-4848-8248-04e7538b1428 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:22.460562 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a81452-0f10-4848-8248-04e7538b1428 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:22.460562 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a81452-0f10-4848-8248-04e7538b1428 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:22.462387 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 767/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 05:42:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:42:22.467496 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98738533-5076-47a3-954e-e5efd2f275a2 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:22.467922 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98738533-5076-47a3-954e-e5efd2f275a2 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:22.468051 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98738533-5076-47a3-954e-e5efd2f275a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:22.491780 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-98738533-5076-47a3-954e-e5efd2f275a2 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:42:22.493520 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-98738533-5076-47a3-954e-e5efd2f275a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:42:22.501719 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 748/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 05:42:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:42:22.663388 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a81452-0f10-4848-8248-04e7538b1428 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:42:22.678657 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a81452-0f10-4848-8248-04e7538b1428 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:42:22.685018 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189ebe23-192c-4328-a3f5-8eb945e51060 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:42:22.685650 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ebe23-192c-4328-a3f5-8eb945e51060 service nova] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:22.685932 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ebe23-192c-4328-a3f5-8eb945e51060 service nova] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:22.686209 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ebe23-192c-4328-a3f5-8eb945e51060 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:22.694266 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189ebe23-192c-4328-a3f5-8eb945e51060 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:42:22.696673 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189ebe23-192c-4328-a3f5-8eb945e51060 service nova] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:42:22.706036 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 768/1516] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:42:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:42:26.022496 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb9569-67c6-4ea8-886c-751d75a2798a None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:26.023035 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb9569-67c6-4ea8-886c-751d75a2798a None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:26.023035 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb9569-67c6-4ea8-886c-751d75a2798a None None] PATH_INFO: `/v3` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:26.025005 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 749/1517] 10.222.0.40 () {54 vars in 925 bytes} [Wed Aug 24 05:42:26 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:42:26.035578 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb7e37-6fad-4c2c-bc47-b09d2a1bb368 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:26.036187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb7e37-6fad-4c2c-bc47-b09d2a1bb368 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:26.036187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb7e37-6fad-4c2c-bc47-b09d2a1bb368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:26.108205 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-98fb7e37-6fad-4c2c-bc47-b09d2a1bb368 None None] MFA Rules not processed for user `6778b7c5ad19484b877494f143050405`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:42:26.109582 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-98fb7e37-6fad-4c2c-bc47-b09d2a1bb368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:42:26.129267 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:42:26.194481 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 769/1518] 10.222.0.40 () {58 vars in 1015 bytes} [Wed Aug 24 05:42:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:42:31.292180 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c77f73-c698-46f0-9f90-6b51682296aa None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:31.292784 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c77f73-c698-46f0-9f90-6b51682296aa None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:31.292784 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c77f73-c698-46f0-9f90-6b51682296aa None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:31.294031 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 750/1519] 10.222.0.15 () {54 vars in 961 bytes} [Wed Aug 24 05:42:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:42:31.299344 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0267cd-b539-4447-8f6d-f0282847bd18 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:31.299808 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0267cd-b539-4447-8f6d-f0282847bd18 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:31.299944 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0267cd-b539-4447-8f6d-f0282847bd18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:31.321636 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-da0267cd-b539-4447-8f6d-f0282847bd18 None None] MFA Rules not processed for user `6778b7c5ad19484b877494f143050405`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:42:31.322864 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-da0267cd-b539-4447-8f6d-f0282847bd18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:42:31.331699 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 770/1520] 10.222.0.15 () {58 vars in 1058 bytes} [Wed Aug 24 05:42:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:42:31.339875 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c77f73-c698-46f0-9f90-6b51682296aa None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:42:31.355619 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c77f73-c698-46f0-9f90-6b51682296aa None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:42:31.360812 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86123342-5964-4822-b111-bc008ea8025e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6778b7c5ad19484b877494f143050405', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:42:31.361421 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86123342-5964-4822-b111-bc008ea8025e service placement] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:31.361710 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86123342-5964-4822-b111-bc008ea8025e service placement] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:31.361987 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86123342-5964-4822-b111-bc008ea8025e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:31.373250 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86123342-5964-4822-b111-bc008ea8025e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:42:31.374867 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86123342-5964-4822-b111-bc008ea8025e service placement] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:42:31.389963 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 751/1521] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:42:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:42:34.002085 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d87007-2d31-4207-aa12-bf45ff42a02e None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:34.002655 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d87007-2d31-4207-aa12-bf45ff42a02e None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:34.002655 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d87007-2d31-4207-aa12-bf45ff42a02e None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:34.003992 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 771/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 05:42:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:42:34.009109 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaff505-e0df-4568-9f11-b397ed62c80c None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:34.009601 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaff505-e0df-4568-9f11-b397ed62c80c None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:34.009712 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaff505-e0df-4568-9f11-b397ed62c80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:34.031992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-cfaff505-e0df-4568-9f11-b397ed62c80c None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:42:34.033237 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-cfaff505-e0df-4568-9f11-b397ed62c80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:42:34.041765 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 752/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 05:42:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:42:34.206441 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d87007-2d31-4207-aa12-bf45ff42a02e None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:42:34.221995 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d87007-2d31-4207-aa12-bf45ff42a02e None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:42:34.228735 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ebefb-8945-4dde-9c2d-e9bd05c247d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:42:34.229385 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ebefb-8945-4dde-9c2d-e9bd05c247d1 service nova] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:34.229674 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ebefb-8945-4dde-9c2d-e9bd05c247d1 service nova] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:34.229975 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ebefb-8945-4dde-9c2d-e9bd05c247d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:34.239038 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6ebefb-8945-4dde-9c2d-e9bd05c247d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:42:34.240689 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6ebefb-8945-4dde-9c2d-e9bd05c247d1 service nova] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:42:34.251709 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 772/1524] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:42:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:42:35.123319 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1526fd2a-3a5c-4398-a8d1-fe58cbcd1290 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:35.123904 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1526fd2a-3a5c-4398-a8d1-fe58cbcd1290 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:35.123904 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1526fd2a-3a5c-4398-a8d1-fe58cbcd1290 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:35.125251 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 753/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 05:42:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:42:35.130767 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97940d6c-165f-4a48-9f97-a3398196ef45 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:35.131265 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97940d6c-165f-4a48-9f97-a3398196ef45 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:35.131395 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97940d6c-165f-4a48-9f97-a3398196ef45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:35.153066 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-97940d6c-165f-4a48-9f97-a3398196ef45 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:42:35.154322 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-97940d6c-165f-4a48-9f97-a3398196ef45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:42:35.162815 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 773/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 05:42:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:42:35.319627 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1526fd2a-3a5c-4398-a8d1-fe58cbcd1290 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:42:35.335672 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1526fd2a-3a5c-4398-a8d1-fe58cbcd1290 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:42:35.342550 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f188dfb-551e-4b11-add4-3a0bb538611a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:42:35.343173 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f188dfb-551e-4b11-add4-3a0bb538611a service nova] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:35.343454 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f188dfb-551e-4b11-add4-3a0bb538611a service nova] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:35.343734 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f188dfb-551e-4b11-add4-3a0bb538611a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:35.357620 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f188dfb-551e-4b11-add4-3a0bb538611a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:42:35.359249 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f188dfb-551e-4b11-add4-3a0bb538611a service nova] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:42:35.371138 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 754/1527] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:42:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:42:39.629514 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cdd435-8715-4c9f-a1fd-fd1c32d6a711 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:39.630321 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cdd435-8715-4c9f-a1fd-fd1c32d6a711 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:39.630321 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cdd435-8715-4c9f-a1fd-fd1c32d6a711 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:39.631465 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 774/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 05:42:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:42:39.638215 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89eb1d-ab7d-4b9d-8d9c-a2d062dcf1c1 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:39.639096 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89eb1d-ab7d-4b9d-8d9c-a2d062dcf1c1 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:39.639096 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89eb1d-ab7d-4b9d-8d9c-a2d062dcf1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:39.660488 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-dd89eb1d-ab7d-4b9d-8d9c-a2d062dcf1c1 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:42:39.661704 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-dd89eb1d-ab7d-4b9d-8d9c-a2d062dcf1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:42:39.671132 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 755/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 05:42:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:42:39.981528 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cdd435-8715-4c9f-a1fd-fd1c32d6a711 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:42:39.983440 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-a9cdd435-8715-4c9f-a1fd-fd1c32d6a711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:42:40.069687 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cdd435-8715-4c9f-a1fd-fd1c32d6a711 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:42:40.073712 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c48a0e-417e-404c-957e-24d7ee54871f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:42:40.074368 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c48a0e-417e-404c-957e-24d7ee54871f service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:40.074652 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c48a0e-417e-404c-957e-24d7ee54871f service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:40.074930 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c48a0e-417e-404c-957e-24d7ee54871f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:40.086749 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c48a0e-417e-404c-957e-24d7ee54871f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:42:40.088937 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c48a0e-417e-404c-957e-24d7ee54871f service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:42:40.100703 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 775/1530] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:42:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:42:42.158074 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ff1b6-4f09-4f37-a4f2-739722bd246b None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:42.158654 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ff1b6-4f09-4f37-a4f2-739722bd246b None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:42.158654 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ff1b6-4f09-4f37-a4f2-739722bd246b None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:42.159965 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 756/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 05:42:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:42:42.167060 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0d4d3-1d14-4d07-aa74-9e065f313707 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:42.167638 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0d4d3-1d14-4d07-aa74-9e065f313707 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:42.167638 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0d4d3-1d14-4d07-aa74-9e065f313707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:42.189016 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-2da0d4d3-1d14-4d07-aa74-9e065f313707 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:42:42.190278 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-2da0d4d3-1d14-4d07-aa74-9e065f313707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:42:42.198997 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 776/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 05:42:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:42:42.499568 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ff1b6-4f09-4f37-a4f2-739722bd246b None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:42:42.501289 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-fc3ff1b6-4f09-4f37-a4f2-739722bd246b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:42:42.515166 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ff1b6-4f09-4f37-a4f2-739722bd246b None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:42:42.519272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9269a6d7-83de-476e-86eb-e006fff69bd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:42:42.520026 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9269a6d7-83de-476e-86eb-e006fff69bd4 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:42.520433 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9269a6d7-83de-476e-86eb-e006fff69bd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:42.520939 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9269a6d7-83de-476e-86eb-e006fff69bd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:42.532374 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9269a6d7-83de-476e-86eb-e006fff69bd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:42:42.534844 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9269a6d7-83de-476e-86eb-e006fff69bd4 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:42:42.544832 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 757/1533] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:42:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:42:46.125991 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6f8c0-2841-414f-b692-5d79a92bdd7f None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:46.126568 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6f8c0-2841-414f-b692-5d79a92bdd7f None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:46.126568 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6f8c0-2841-414f-b692-5d79a92bdd7f None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:46.128186 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 777/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 05:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:42:46.134217 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ffb98-d0cf-4f4b-964a-a87978678a15 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:46.134663 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ffb98-d0cf-4f4b-964a-a87978678a15 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:46.134777 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ffb98-d0cf-4f4b-964a-a87978678a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:46.142426 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=81 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:42:46.161686 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-f87ffb98-d0cf-4f4b-964a-a87978678a15 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:42:46.164120 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-f87ffb98-d0cf-4f4b-964a-a87978678a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:42:46.172792 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 758/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 05:42:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:42:46.772672 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f6f8c0-2841-414f-b692-5d79a92bdd7f None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:42:46.775391 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-92f6f8c0-2841-414f-b692-5d79a92bdd7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:42:46.855474 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:42:46.872569 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f6f8c0-2841-414f-b692-5d79a92bdd7f None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:42:46.876659 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b105457e-a18f-43b3-8944-3fc78093bfb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:42:46.877326 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b105457e-a18f-43b3-8944-3fc78093bfb0 service cinder] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:46.877593 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b105457e-a18f-43b3-8944-3fc78093bfb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:46.877873 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b105457e-a18f-43b3-8944-3fc78093bfb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:46.890812 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b105457e-a18f-43b3-8944-3fc78093bfb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:42:46.892308 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b105457e-a18f-43b3-8944-3fc78093bfb0 service cinder] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:42:46.910193 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 778/1536] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:42:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:42:48.911401 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0af32b-49f5-4015-ac91-7658f29155eb None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:48.911977 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0af32b-49f5-4015-ac91-7658f29155eb None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:48.911977 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0af32b-49f5-4015-ac91-7658f29155eb None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:48.913813 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 759/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 05:42:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:42:48.920367 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7b67f-52ad-4ed6-a9dd-78ff98a472f8 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:48.920910 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7b67f-52ad-4ed6-a9dd-78ff98a472f8 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:48.921030 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7b67f-52ad-4ed6-a9dd-78ff98a472f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:48.943067 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-c3f7b67f-52ad-4ed6-a9dd-78ff98a472f8 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:42:48.944308 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-c3f7b67f-52ad-4ed6-a9dd-78ff98a472f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:42:48.953455 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 779/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 05:42:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:42:49.570845 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0af32b-49f5-4015-ac91-7658f29155eb None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:42:49.572780 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-5a0af32b-49f5-4015-ac91-7658f29155eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:42:49.595227 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0af32b-49f5-4015-ac91-7658f29155eb None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:42:49.599703 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4343f176-d586-4911-bc97-3db2b2e96eb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:42:49.600516 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4343f176-d586-4911-bc97-3db2b2e96eb5 service cinder] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:49.600824 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4343f176-d586-4911-bc97-3db2b2e96eb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:49.601256 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4343f176-d586-4911-bc97-3db2b2e96eb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:49.612721 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4343f176-d586-4911-bc97-3db2b2e96eb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:42:49.614227 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4343f176-d586-4911-bc97-3db2b2e96eb5 service cinder] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:42:49.625579 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 760/1539] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:42:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:42:58.938033 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=11 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:42:59.620693 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=8 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:19.483275 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc4691-a1e1-414e-94bf-bb9d0b47c8f6 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:19.493739 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc4691-a1e1-414e-94bf-bb9d0b47c8f6 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:19.493739 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc4691-a1e1-414e-94bf-bb9d0b47c8f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:19.510805 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-c9fc4691-a1e1-414e-94bf-bb9d0b47c8f6 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:19.512021 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-c9fc4691-a1e1-414e-94bf-bb9d0b47c8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:19.525671 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 780/1540] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:19.534415 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92c28d-806c-48e6-a63e-83e55fc79952 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:19.534992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92c28d-806c-48e6-a63e-83e55fc79952 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:19.535098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92c28d-806c-48e6-a63e-83e55fc79952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:19.565484 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-2a92c28d-806c-48e6-a63e-83e55fc79952 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:19.566704 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-2a92c28d-806c-48e6-a63e-83e55fc79952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:19.575845 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 761/1541] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:19.580292 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc4691-a1e1-414e-94bf-bb9d0b47c8f6 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:19.596011 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc4691-a1e1-414e-94bf-bb9d0b47c8f6 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:19.606266 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bfd2ad-499c-4f4f-b0d3-e399aa833145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:19.607090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bfd2ad-499c-4f4f-b0d3-e399aa833145 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:19.607372 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bfd2ad-499c-4f4f-b0d3-e399aa833145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:19.607649 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bfd2ad-499c-4f4f-b0d3-e399aa833145 admin admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:19.608308 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bfd2ad-499c-4f4f-b0d3-e399aa833145 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:44:19.608645 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bfd2ad-499c-4f4f-b0d3-e399aa833145 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:19.610285 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bfd2ad-499c-4f4f-b0d3-e399aa833145 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:19.629028 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 781/1542] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:44:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:44:19.637436 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92c28d-806c-48e6-a63e-83e55fc79952 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:19.647683 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92c28d-806c-48e6-a63e-83e55fc79952 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:19.655608 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aefe0fa-cba5-4e5d-a2ae-15a346357209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:19.656226 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aefe0fa-cba5-4e5d-a2ae-15a346357209 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:19.656534 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aefe0fa-cba5-4e5d-a2ae-15a346357209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:19.656811 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aefe0fa-cba5-4e5d-a2ae-15a346357209 admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:19.657455 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aefe0fa-cba5-4e5d-a2ae-15a346357209 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:19.659578 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aefe0fa-cba5-4e5d-a2ae-15a346357209 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:19.832535 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 762/1543] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:44:19 2022] POST /identity/v3/projects => generated 373 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:44:19.832864 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bfd2ad-499c-4f4f-b0d3-e399aa833145 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:19.841535 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bfd2ad-499c-4f4f-b0d3-e399aa833145 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:19.914488 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0363eed-9e04-4802-9277-f8c6a9e6ac5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:19.915391 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0363eed-9e04-4802-9277-f8c6a9e6ac5f admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:19.915677 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0363eed-9e04-4802-9277-f8c6a9e6ac5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:19.915949 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0363eed-9e04-4802-9277-f8c6a9e6ac5f admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:19.917228 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0363eed-9e04-4802-9277-f8c6a9e6ac5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:19.919181 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0363eed-9e04-4802-9277-f8c6a9e6ac5f admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:20.001989 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 782/1544] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:44:19 2022] POST /identity/v3/users => generated 287 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:44:20.007344 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aefe0fa-cba5-4e5d-a2ae-15a346357209 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:20.022929 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aefe0fa-cba5-4e5d-a2ae-15a346357209 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:20.037952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa8f2d2-d2d2-45d7-b244-6d0b5889822b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:20.038722 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa8f2d2-d2d2-45d7-b244-6d0b5889822b admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:20.039006 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa8f2d2-d2d2-45d7-b244-6d0b5889822b admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:20.039284 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa8f2d2-d2d2-45d7-b244-6d0b5889822b admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:20.040055 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa8f2d2-d2d2-45d7-b244-6d0b5889822b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:20.042198 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa8f2d2-d2d2-45d7-b244-6d0b5889822b admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:20.061633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 763/1545] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:44:20 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:44:20.066358 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0363eed-9e04-4802-9277-f8c6a9e6ac5f admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:20.082503 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0363eed-9e04-4802-9277-f8c6a9e6ac5f admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:20.090475 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283f569a-5c19-42e8-afe9-2bffea1c78d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:20.091024 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f569a-5c19-42e8-afe9-2bffea1c78d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:20.091312 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f569a-5c19-42e8-afe9-2bffea1c78d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:20.091596 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f569a-5c19-42e8-afe9-2bffea1c78d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:20.092262 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283f569a-5c19-42e8-afe9-2bffea1c78d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:20.094052 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283f569a-5c19-42e8-afe9-2bffea1c78d5 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:20.113867 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 783/1546] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:44:20 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:44:20.122036 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa8f2d2-d2d2-45d7-b244-6d0b5889822b admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:20.133808 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa8f2d2-d2d2-45d7-b244-6d0b5889822b admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:20.143286 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dfef85-4d10-4072-9155-ee1b8f49e004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:20.144016 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dfef85-4d10-4072-9155-ee1b8f49e004 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:20.144303 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dfef85-4d10-4072-9155-ee1b8f49e004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:20.144623 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dfef85-4d10-4072-9155-ee1b8f49e004 admin admin] PATH_INFO: `/v3/projects/69d11893b9624f6bbf8c8eeda08d4858/users/e9cd9823dd8b4195b6f782b8cf391a07/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:20.170148 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dfef85-4d10-4072-9155-ee1b8f49e004 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69d11893b9624f6bbf8c8eeda08d4858, user_id=e9cd9823dd8b4195b6f782b8cf391a07, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:20.173370 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dfef85-4d10-4072-9155-ee1b8f49e004 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:20.283210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 764/1547] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:44:20 2022] PUT /identity/v3/projects/69d11893b9624f6bbf8c8eeda08d4858/users/e9cd9823dd8b4195b6f782b8cf391a07/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:44:20.330566 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904e604-7548-4104-bb6f-1d5a64364212 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:20.330849 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904e604-7548-4104-bb6f-1d5a64364212 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:20.331127 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904e604-7548-4104-bb6f-1d5a64364212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:20.374503 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-f904e604-7548-4104-bb6f-1d5a64364212 None None] MFA Rules not processed for user `e9cd9823dd8b4195b6f782b8cf391a07`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:20.375757 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-f904e604-7548-4104-bb6f-1d5a64364212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:20.472436 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6bf34-e565-4fbe-b6b6-502bb76cd6cb None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:20.472800 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6bf34-e565-4fbe-b6b6-502bb76cd6cb None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:20.473019 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6bf34-e565-4fbe-b6b6-502bb76cd6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:20.488510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 784/1548] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:20.491558 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6e48b-d8b1-4c13-bed2-ede94f76b8c3 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:20.492148 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6e48b-d8b1-4c13-bed2-ede94f76b8c3 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:20.492686 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6e48b-d8b1-4c13-bed2-ede94f76b8c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:20.519718 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-55c6bf34-e565-4fbe-b6b6-502bb76cd6cb None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:20.523746 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-55c6bf34-e565-4fbe-b6b6-502bb76cd6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:20.528724 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-26a6e48b-d8b1-4c13-bed2-ede94f76b8c3 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:20.530054 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-26a6e48b-d8b1-4c13-bed2-ede94f76b8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:20.636948 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 765/1549] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:20.639591 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417bc7da-0d2a-477f-a644-21f4095fae2a None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:20.647976 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417bc7da-0d2a-477f-a644-21f4095fae2a None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:20.648340 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417bc7da-0d2a-477f-a644-21f4095fae2a None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:20.649919 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 766/1550] 10.222.0.15 () {54 vars in 962 bytes} [Wed Aug 24 05:44:20 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:44:20.652469 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c30f7-375e-462b-9632-646f816248e8 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:20.652765 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c30f7-375e-462b-9632-646f816248e8 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:20.655093 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c30f7-375e-462b-9632-646f816248e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:20.665650 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:20.681516 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-b82c30f7-375e-462b-9632-646f816248e8 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:20.682752 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-b82c30f7-375e-462b-9632-646f816248e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:20.686825 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 785/1551] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:20.689479 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a437e4-568e-4e95-ac4e-5500c30aba42 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:20.690097 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a437e4-568e-4e95-ac4e-5500c30aba42 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:20.690601 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a437e4-568e-4e95-ac4e-5500c30aba42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:20.691801 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 767/1552] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:20.696699 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc231cf-8298-4d90-9657-b01b41687cf3 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:20.696699 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc231cf-8298-4d90-9657-b01b41687cf3 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:20.696699 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc231cf-8298-4d90-9657-b01b41687cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:20.720343 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-9bc231cf-8298-4d90-9657-b01b41687cf3 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:20.723905 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-9bc231cf-8298-4d90-9657-b01b41687cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:20.725860 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-33a437e4-568e-4e95-ac4e-5500c30aba42 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:20.727121 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-33a437e4-568e-4e95-ac4e-5500c30aba42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:20.731050 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 768/1553] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:20.732326 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c64fef-d76c-4a28-9fb8-86c979335f4c None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:20.736166 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 786/1554] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:20.737492 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14806e27-8a66-42b1-8f85-35c4ecef9703 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:20.737778 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14806e27-8a66-42b1-8f85-35c4ecef9703 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:20.738098 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14806e27-8a66-42b1-8f85-35c4ecef9703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:20.740727 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c64fef-d76c-4a28-9fb8-86c979335f4c None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:20.741080 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c64fef-d76c-4a28-9fb8-86c979335f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:20.762229 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-f9c64fef-d76c-4a28-9fb8-86c979335f4c None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:20.764868 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-f9c64fef-d76c-4a28-9fb8-86c979335f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:20.776798 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 769/1555] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:20.778265 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7c743-cb69-4429-b0d4-23b9abbdd45c None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:20.778594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7c743-cb69-4429-b0d4-23b9abbdd45c None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:20.778902 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7c743-cb69-4429-b0d4-23b9abbdd45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:20.784826 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-14806e27-8a66-42b1-8f85-35c4ecef9703 None None] MFA Rules not processed for user `fbecffb800584c5786813accd061421b`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:20.786185 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-14806e27-8a66-42b1-8f85-35c4ecef9703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:20.800959 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-0ef7c743-cb69-4429-b0d4-23b9abbdd45c None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:20.802698 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-0ef7c743-cb69-4429-b0d4-23b9abbdd45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:20.822351 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 770/1556] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:20.822351 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00672b9f-56db-4ab2-b96f-07ac5900124d None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:20.822351 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00672b9f-56db-4ab2-b96f-07ac5900124d None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:20.822351 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00672b9f-56db-4ab2-b96f-07ac5900124d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:20.835605 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-00672b9f-56db-4ab2-b96f-07ac5900124d None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:20.844543 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-00672b9f-56db-4ab2-b96f-07ac5900124d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:20.846953 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 771/1557] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:20.851351 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545a380-6b44-46df-9037-cefb61c047ce None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:20.851351 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545a380-6b44-46df-9037-cefb61c047ce None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:20.851351 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545a380-6b44-46df-9037-cefb61c047ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:20.870992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-2545a380-6b44-46df-9037-cefb61c047ce None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:20.873310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-2545a380-6b44-46df-9037-cefb61c047ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:20.880473 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 787/1558] 10.222.0.15 () {58 vars in 1059 bytes} [Wed Aug 24 05:44:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:20.888606 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 772/1559] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:20.888606 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21812630-ddfc-4ba4-9a00-379dcd6127c2 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:20.888606 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21812630-ddfc-4ba4-9a00-379dcd6127c2 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:20.888969 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc542c-f9df-4374-b7f0-57f324d48fba None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:20.889070 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21812630-ddfc-4ba4-9a00-379dcd6127c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:20.889162 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc542c-f9df-4374-b7f0-57f324d48fba None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:20.889162 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc542c-f9df-4374-b7f0-57f324d48fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:20.907084 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-21812630-ddfc-4ba4-9a00-379dcd6127c2 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:20.908314 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-21812630-ddfc-4ba4-9a00-379dcd6127c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:20.908492 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-13bc542c-f9df-4374-b7f0-57f324d48fba None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:20.909572 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-13bc542c-f9df-4374-b7f0-57f324d48fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:20.918485 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 788/1560] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:20.922889 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 773/1561] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:20.923043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc542c-f9df-4374-b7f0-57f324d48fba None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:20.923148 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276b6ba-2bed-4788-9593-9ccce8dc64ec None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:20.923148 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276b6ba-2bed-4788-9593-9ccce8dc64ec None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:20.923148 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276b6ba-2bed-4788-9593-9ccce8dc64ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:20.939006 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc542c-f9df-4374-b7f0-57f324d48fba None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:20.941856 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-9276b6ba-2bed-4788-9593-9ccce8dc64ec None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:20.943849 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-9276b6ba-2bed-4788-9593-9ccce8dc64ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:20.944546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1289e3-8863-4be5-95a7-92b2c0ae32da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:20.945164 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1289e3-8863-4be5-95a7-92b2c0ae32da admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:20.945445 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1289e3-8863-4be5-95a7-92b2c0ae32da admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:20.945720 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1289e3-8863-4be5-95a7-92b2c0ae32da admin admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:20.947175 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1289e3-8863-4be5-95a7-92b2c0ae32da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:44:20.947175 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1289e3-8863-4be5-95a7-92b2c0ae32da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:20.948492 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1289e3-8863-4be5-95a7-92b2c0ae32da admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:20.952190 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 774/1562] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:20.953149 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9276b6ba-2bed-4788-9593-9ccce8dc64ec None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:20.962071 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 789/1563] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:44:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:44:20.963071 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1289e3-8863-4be5-95a7-92b2c0ae32da admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:20.969263 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9276b6ba-2bed-4788-9593-9ccce8dc64ec None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:20.974393 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1289e3-8863-4be5-95a7-92b2c0ae32da admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:20.977453 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade9221-246e-4f51-938f-1acc94458984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:20.978079 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade9221-246e-4f51-938f-1acc94458984 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:20.978357 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade9221-246e-4f51-938f-1acc94458984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:20.978629 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade9221-246e-4f51-938f-1acc94458984 admin admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:20.979280 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade9221-246e-4f51-938f-1acc94458984 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:44:20.979398 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f87694-53a4-41d2-aaef-6df6e49cb93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:20.979630 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade9221-246e-4f51-938f-1acc94458984 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:20.979990 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f87694-53a4-41d2-aaef-6df6e49cb93c admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:20.980272 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f87694-53a4-41d2-aaef-6df6e49cb93c admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:20.980583 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f87694-53a4-41d2-aaef-6df6e49cb93c admin admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:20.984792 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: WARNING py.warnings [None req-7ade9221-246e-4f51-938f-1acc94458984 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 05:44:20.984792 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: warnings.warn(msg) Aug 24 05:44:20.984792 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: Aug 24 05:44:20.985063 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f87694-53a4-41d2-aaef-6df6e49cb93c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:44:20.985152 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade9221-246e-4f51-938f-1acc94458984 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:20.985237 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f87694-53a4-41d2-aaef-6df6e49cb93c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:20.985237 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f87694-53a4-41d2-aaef-6df6e49cb93c admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:20.995799 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 775/1564] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:44:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:44:21.003963 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 790/1565] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:44:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:44:21.003963 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f87694-53a4-41d2-aaef-6df6e49cb93c admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:21.004358 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e574da1-0e79-42fb-97fa-23f83d848230 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:21.004358 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e574da1-0e79-42fb-97fa-23f83d848230 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:21.004358 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e574da1-0e79-42fb-97fa-23f83d848230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:21.018877 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f87694-53a4-41d2-aaef-6df6e49cb93c admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:21.023498 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-0e574da1-0e79-42fb-97fa-23f83d848230 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:21.024815 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-0e574da1-0e79-42fb-97fa-23f83d848230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:21.032986 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 776/1566] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:21.034076 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e574da1-0e79-42fb-97fa-23f83d848230 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:21.034930 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a1c830-7643-4ecd-88c5-aa9fbcf04f53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:21.035774 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1c830-7643-4ecd-88c5-aa9fbcf04f53 service nova] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:21.036476 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1c830-7643-4ecd-88c5-aa9fbcf04f53 service nova] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:21.037556 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1c830-7643-4ecd-88c5-aa9fbcf04f53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:21.044422 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e574da1-0e79-42fb-97fa-23f83d848230 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:21.046806 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a1c830-7643-4ecd-88c5-aa9fbcf04f53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:21.048429 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a1c830-7643-4ecd-88c5-aa9fbcf04f53 service nova] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:21.049510 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bdc7dd-7cc1-4188-9927-119cb031806b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:21.050135 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bdc7dd-7cc1-4188-9927-119cb031806b admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:21.050414 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bdc7dd-7cc1-4188-9927-119cb031806b admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:21.050711 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bdc7dd-7cc1-4188-9927-119cb031806b admin admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:21.051427 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bdc7dd-7cc1-4188-9927-119cb031806b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:44:21.052382 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bdc7dd-7cc1-4188-9927-119cb031806b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:21.055210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bdc7dd-7cc1-4188-9927-119cb031806b admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:21.060224 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 791/1567] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:44:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:44:21.062168 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dab393-522f-402d-9b53-6c3300999634 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:21.062168 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dab393-522f-402d-9b53-6c3300999634 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:21.062837 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dab393-522f-402d-9b53-6c3300999634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:21.077537 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 777/1568] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:44:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:44:21.078404 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bdc7dd-7cc1-4188-9927-119cb031806b admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:21.091123 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-f9dab393-522f-402d-9b53-6c3300999634 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:21.091813 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:21.093385 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-f9dab393-522f-402d-9b53-6c3300999634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:21.097730 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bdc7dd-7cc1-4188-9927-119cb031806b admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:21.102985 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30065ed6-1c80-4837-a530-670c0e2968bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:21.103257 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 792/1569] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:21.105237 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30065ed6-1c80-4837-a530-670c0e2968bc admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:21.105362 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dab393-522f-402d-9b53-6c3300999634 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:21.105487 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30065ed6-1c80-4837-a530-670c0e2968bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:21.105487 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30065ed6-1c80-4837-a530-670c0e2968bc admin admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:21.106328 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30065ed6-1c80-4837-a530-670c0e2968bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:44:21.106823 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30065ed6-1c80-4837-a530-670c0e2968bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:21.109610 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30065ed6-1c80-4837-a530-670c0e2968bc admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:21.115725 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dab393-522f-402d-9b53-6c3300999634 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:21.120836 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca267737-a4c7-4287-9225-679f5ba968f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:21.121468 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca267737-a4c7-4287-9225-679f5ba968f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:21.121748 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca267737-a4c7-4287-9225-679f5ba968f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:21.122022 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca267737-a4c7-4287-9225-679f5ba968f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:21.122691 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca267737-a4c7-4287-9225-679f5ba968f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:21.123905 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: WARNING py.warnings [None req-ca267737-a4c7-4287-9225-679f5ba968f0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 05:44:21.123905 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: warnings.warn(msg) Aug 24 05:44:21.123905 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: Aug 24 05:44:21.134308 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 778/1570] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:44:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:44:21.142659 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca267737-a4c7-4287-9225-679f5ba968f0 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:21.142824 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30065ed6-1c80-4837-a530-670c0e2968bc admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:21.148700 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30065ed6-1c80-4837-a530-670c0e2968bc admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:21.153686 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f98d034-7ae9-4fe8-851e-7ba0710a7440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:21.154393 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f98d034-7ae9-4fe8-851e-7ba0710a7440 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:21.154736 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f98d034-7ae9-4fe8-851e-7ba0710a7440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:21.155078 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f98d034-7ae9-4fe8-851e-7ba0710a7440 admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:21.155799 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f98d034-7ae9-4fe8-851e-7ba0710a7440 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:21.158296 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f98d034-7ae9-4fe8-851e-7ba0710a7440 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:21.248374 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 793/1571] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:44:21 2022] POST /identity/v3/projects => generated 387 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:44:21.248374 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca267737-a4c7-4287-9225-679f5ba968f0 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:21.303368 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:21.317398 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 779/1572] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:44:21 2022] POST /identity/v3/projects => generated 379 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:44:21.318870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f98d034-7ae9-4fe8-851e-7ba0710a7440 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:21.326892 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca267737-a4c7-4287-9225-679f5ba968f0 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:21.391363 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f98d034-7ae9-4fe8-851e-7ba0710a7440 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:21.396341 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7c4fe0-4459-446b-98d8-b01cdef31a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:21.397080 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c4fe0-4459-446b-98d8-b01cdef31a60 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:21.397359 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c4fe0-4459-446b-98d8-b01cdef31a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:21.397638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c4fe0-4459-446b-98d8-b01cdef31a60 admin admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:21.398288 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7c4fe0-4459-446b-98d8-b01cdef31a60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:44:21.398592 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7c4fe0-4459-446b-98d8-b01cdef31a60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:21.400220 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7c4fe0-4459-446b-98d8-b01cdef31a60 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:21.405177 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fce8d2-8405-4477-a970-4e3a548fd532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:21.405848 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fce8d2-8405-4477-a970-4e3a548fd532 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:21.406131 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fce8d2-8405-4477-a970-4e3a548fd532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:21.406409 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fce8d2-8405-4477-a970-4e3a548fd532 admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:21.407052 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fce8d2-8405-4477-a970-4e3a548fd532 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:21.409018 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fce8d2-8405-4477-a970-4e3a548fd532 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:21.420299 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 780/1573] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:44:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:44:21.421205 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7c4fe0-4459-446b-98d8-b01cdef31a60 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:21.432825 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7c4fe0-4459-446b-98d8-b01cdef31a60 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:21.437939 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b2676-0911-413f-bdd8-bb4d8652beef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:21.438551 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b2676-0911-413f-bdd8-bb4d8652beef admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:21.438833 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b2676-0911-413f-bdd8-bb4d8652beef admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:21.439109 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b2676-0911-413f-bdd8-bb4d8652beef admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:21.439744 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4b2676-0911-413f-bdd8-bb4d8652beef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:21.441696 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4b2676-0911-413f-bdd8-bb4d8652beef admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:21.513549 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 794/1574] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:44:21 2022] POST /identity/v3/projects => generated 383 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:44:21.514595 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fce8d2-8405-4477-a970-4e3a548fd532 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:21.547857 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 781/1575] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:44:21 2022] POST /identity/v3/projects => generated 363 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:44:21.549042 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b2676-0911-413f-bdd8-bb4d8652beef admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:21.585200 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fce8d2-8405-4477-a970-4e3a548fd532 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:21.618218 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b2676-0911-413f-bdd8-bb4d8652beef admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:21.623108 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be88074f-0451-4763-8e90-bac6a33dcfc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:21.623730 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be88074f-0451-4763-8e90-bac6a33dcfc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:21.624007 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be88074f-0451-4763-8e90-bac6a33dcfc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:21.624277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be88074f-0451-4763-8e90-bac6a33dcfc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:21.624963 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be88074f-0451-4763-8e90-bac6a33dcfc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:21.627086 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be88074f-0451-4763-8e90-bac6a33dcfc5 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:21.649579 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c223479-6500-4c47-b152-42d6dcad37f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:21.650117 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c223479-6500-4c47-b152-42d6dcad37f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:21.650405 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c223479-6500-4c47-b152-42d6dcad37f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:21.650682 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c223479-6500-4c47-b152-42d6dcad37f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:21.651336 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c223479-6500-4c47-b152-42d6dcad37f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:44:21.651645 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c223479-6500-4c47-b152-42d6dcad37f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:21.653327 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c223479-6500-4c47-b152-42d6dcad37f3 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:21.665651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 795/1576] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:44:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:44:21.666626 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c223479-6500-4c47-b152-42d6dcad37f3 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:21.678325 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c223479-6500-4c47-b152-42d6dcad37f3 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:21.683202 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce74650b-5a80-46ec-8a7f-a6135bf976d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:21.683827 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74650b-5a80-46ec-8a7f-a6135bf976d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:21.684309 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74650b-5a80-46ec-8a7f-a6135bf976d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:21.684462 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74650b-5a80-46ec-8a7f-a6135bf976d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:21.685156 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce74650b-5a80-46ec-8a7f-a6135bf976d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:21.687113 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce74650b-5a80-46ec-8a7f-a6135bf976d1 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:21.768621 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 782/1577] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:44:21 2022] POST /identity/v3/projects => generated 393 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:44:21.770125 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13ad14-6f5d-4fed-bb9c-9efd21f16603 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:21.770435 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13ad14-6f5d-4fed-bb9c-9efd21f16603 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:21.770945 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13ad14-6f5d-4fed-bb9c-9efd21f16603 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:21.772190 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 783/1578] 10.222.0.15 () {54 vars in 912 bytes} [Wed Aug 24 05:44:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:44:21.772972 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f13ad14-6f5d-4fed-bb9c-9efd21f16603 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:21.803729 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:21.868978 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f13ad14-6f5d-4fed-bb9c-9efd21f16603 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:21.876681 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 796/1579] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:44:21 2022] POST /identity/v3/users => generated 294 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:44:21.877527 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce74650b-5a80-46ec-8a7f-a6135bf976d1 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:21.878311 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9041c048-b36e-49dc-ba61-b690c5c6a0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:21.879509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9041c048-b36e-49dc-ba61-b690c5c6a0ae admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:21.880671 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9041c048-b36e-49dc-ba61-b690c5c6a0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:21.881418 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9041c048-b36e-49dc-ba61-b690c5c6a0ae admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:21.882478 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9041c048-b36e-49dc-ba61-b690c5c6a0ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:21.883459 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:21.884834 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: WARNING py.warnings [None req-9041c048-b36e-49dc-ba61-b690c5c6a0ae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 05:44:21.884834 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: warnings.warn(msg) Aug 24 05:44:21.884834 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: Aug 24 05:44:21.888165 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9041c048-b36e-49dc-ba61-b690c5c6a0ae admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:21.889823 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce74650b-5a80-46ec-8a7f-a6135bf976d1 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:21.895935 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86087507-f40f-460b-9bbd-38e59e37856e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:21.898085 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86087507-f40f-460b-9bbd-38e59e37856e admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:21.898085 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86087507-f40f-460b-9bbd-38e59e37856e admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:21.898085 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86087507-f40f-460b-9bbd-38e59e37856e admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:21.898085 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86087507-f40f-460b-9bbd-38e59e37856e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:21.900247 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86087507-f40f-460b-9bbd-38e59e37856e admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:21.997698 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 784/1580] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:44:21 2022] POST /identity/v3/users => generated 290 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:44:21.998583 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9041c048-b36e-49dc-ba61-b690c5c6a0ae admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:22.010521 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9041c048-b36e-49dc-ba61-b690c5c6a0ae admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:22.015390 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace9420b-2e4d-4646-9054-f9a08f3697af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:22.016018 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9420b-2e4d-4646-9054-f9a08f3697af admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:22.016295 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9420b-2e4d-4646-9054-f9a08f3697af admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:22.016629 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9420b-2e4d-4646-9054-f9a08f3697af admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:22.017248 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace9420b-2e4d-4646-9054-f9a08f3697af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:22.019185 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace9420b-2e4d-4646-9054-f9a08f3697af admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:22.039700 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 797/1581] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:44:21 2022] POST /identity/v3/projects => generated 383 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:44:22.041473 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86087507-f40f-460b-9bbd-38e59e37856e admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:22.107225 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86087507-f40f-460b-9bbd-38e59e37856e admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:22.112161 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4a55d6-37b6-49aa-bcc6-ebd22a9052fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:22.112821 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a55d6-37b6-49aa-bcc6-ebd22a9052fd admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:22.113107 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a55d6-37b6-49aa-bcc6-ebd22a9052fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:22.113393 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a55d6-37b6-49aa-bcc6-ebd22a9052fd admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:22.114767 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4a55d6-37b6-49aa-bcc6-ebd22a9052fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:22.117292 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4a55d6-37b6-49aa-bcc6-ebd22a9052fd admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:22.132516 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 785/1582] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:44:21 2022] POST /identity/v3/users => generated 292 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:44:22.133454 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace9420b-2e4d-4646-9054-f9a08f3697af admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:22.143768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace9420b-2e4d-4646-9054-f9a08f3697af admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:22.149159 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff58fe4-a873-4d1f-96df-6cb169f7d335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:22.149778 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff58fe4-a873-4d1f-96df-6cb169f7d335 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:22.150059 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff58fe4-a873-4d1f-96df-6cb169f7d335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:22.150390 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff58fe4-a873-4d1f-96df-6cb169f7d335 admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:22.150968 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff58fe4-a873-4d1f-96df-6cb169f7d335 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:22.152942 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff58fe4-a873-4d1f-96df-6cb169f7d335 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:22.276939 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 798/1583] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:44:22 2022] POST /identity/v3/users => generated 282 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:44:22.277804 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4a55d6-37b6-49aa-bcc6-ebd22a9052fd admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:22.288694 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4a55d6-37b6-49aa-bcc6-ebd22a9052fd admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:22.294234 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e43e613-693d-4923-8b2c-6b97cc6c3c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:22.294897 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43e613-693d-4923-8b2c-6b97cc6c3c48 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:22.295181 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43e613-693d-4923-8b2c-6b97cc6c3c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:22.295456 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43e613-693d-4923-8b2c-6b97cc6c3c48 admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:22.296182 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e43e613-693d-4923-8b2c-6b97cc6c3c48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:22.298297 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e43e613-693d-4923-8b2c-6b97cc6c3c48 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:22.311390 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 786/1584] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:44:22 2022] POST /identity/v3/projects => generated 371 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:44:22.312703 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50913be2-b994-41bd-add2-b073d7420698 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:22.312995 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50913be2-b994-41bd-add2-b073d7420698 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:22.321483 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50913be2-b994-41bd-add2-b073d7420698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:22.347503 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-50913be2-b994-41bd-add2-b073d7420698 None None] MFA Rules not processed for user `fbecffb800584c5786813accd061421b`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:22.350546 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-50913be2-b994-41bd-add2-b073d7420698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:22.413323 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 799/1585] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:44:22 2022] POST /identity/v3/users => generated 297 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:44:22.414681 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e43e613-693d-4923-8b2c-6b97cc6c3c48 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:22.428758 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e43e613-693d-4923-8b2c-6b97cc6c3c48 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:22.432098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 787/1586] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Aug 24 05:44:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:22.433006 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50913be2-b994-41bd-add2-b073d7420698 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:22.507483 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50913be2-b994-41bd-add2-b073d7420698 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:22.511445 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bdfe9b-3cd6-48cf-a86b-194e1e766bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:22.512248 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bdfe9b-3cd6-48cf-a86b-194e1e766bcf admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:22.512493 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c808b-8d9c-452c-abcc-c49b7be4233b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:22.512698 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bdfe9b-3cd6-48cf-a86b-194e1e766bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:22.512866 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bdfe9b-3cd6-48cf-a86b-194e1e766bcf admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:22.513127 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c808b-8d9c-452c-abcc-c49b7be4233b admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:22.513501 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c808b-8d9c-452c-abcc-c49b7be4233b admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:22.513660 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bdfe9b-3cd6-48cf-a86b-194e1e766bcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:22.514192 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c808b-8d9c-452c-abcc-c49b7be4233b admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:22.514336 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10c808b-8d9c-452c-abcc-c49b7be4233b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:22.515166 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bdfe9b-3cd6-48cf-a86b-194e1e766bcf admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:22.515969 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10c808b-8d9c-452c-abcc-c49b7be4233b admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:22.531142 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 800/1587] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:44:22.531142 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bdfe9b-3cd6-48cf-a86b-194e1e766bcf admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:22.531591 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 788/1588] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:44:22.531591 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c808b-8d9c-452c-abcc-c49b7be4233b admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:22.540419 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:22.543095 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bdfe9b-3cd6-48cf-a86b-194e1e766bcf admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:22.548116 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c808b-8d9c-452c-abcc-c49b7be4233b admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:22.548336 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398d2225-63da-4285-99db-4eb0c1eeb296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:22.548336 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398d2225-63da-4285-99db-4eb0c1eeb296 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:22.548673 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398d2225-63da-4285-99db-4eb0c1eeb296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:22.548819 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398d2225-63da-4285-99db-4eb0c1eeb296 admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:22.549331 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398d2225-63da-4285-99db-4eb0c1eeb296 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:22.550701 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b151005-2d20-46db-b30c-de21052d6d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:22.551262 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398d2225-63da-4285-99db-4eb0c1eeb296 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:22.551412 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b151005-2d20-46db-b30c-de21052d6d17 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:22.552221 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b151005-2d20-46db-b30c-de21052d6d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:22.552636 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b151005-2d20-46db-b30c-de21052d6d17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:22.553303 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b151005-2d20-46db-b30c-de21052d6d17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:22.555110 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b151005-2d20-46db-b30c-de21052d6d17 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:22.566614 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 789/1589] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:44:22.568484 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b151005-2d20-46db-b30c-de21052d6d17 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:22.581669 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b151005-2d20-46db-b30c-de21052d6d17 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:22.586993 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed63e26d-237f-4fa3-ac15-e7c7aede04b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:22.587609 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed63e26d-237f-4fa3-ac15-e7c7aede04b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:22.597130 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed63e26d-237f-4fa3-ac15-e7c7aede04b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:22.597130 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed63e26d-237f-4fa3-ac15-e7c7aede04b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:22.597130 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed63e26d-237f-4fa3-ac15-e7c7aede04b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:22.597130 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed63e26d-237f-4fa3-ac15-e7c7aede04b5 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:22.600911 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 790/1590] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:44:22.601802 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed63e26d-237f-4fa3-ac15-e7c7aede04b5 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:22.611869 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed63e26d-237f-4fa3-ac15-e7c7aede04b5 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:22.617626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff0be5d-c4bd-40c1-8a7f-f2180534aa4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:22.618281 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff0be5d-c4bd-40c1-8a7f-f2180534aa4a admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:22.618561 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff0be5d-c4bd-40c1-8a7f-f2180534aa4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:22.618832 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff0be5d-c4bd-40c1-8a7f-f2180534aa4a admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:22.619497 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff0be5d-c4bd-40c1-8a7f-f2180534aa4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:22.621691 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff0be5d-c4bd-40c1-8a7f-f2180534aa4a admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:22.676204 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 801/1591] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:44:22 2022] POST /identity/v3/users => generated 292 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:44:22.678149 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398d2225-63da-4285-99db-4eb0c1eeb296 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:22.688106 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398d2225-63da-4285-99db-4eb0c1eeb296 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:22.693486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ca1db-4470-4114-8c97-519e9c8689eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:22.694185 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ca1db-4470-4114-8c97-519e9c8689eb admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:22.694504 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ca1db-4470-4114-8c97-519e9c8689eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:22.694822 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ca1db-4470-4114-8c97-519e9c8689eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:22.695516 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114ca1db-4470-4114-8c97-519e9c8689eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:22.697352 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114ca1db-4470-4114-8c97-519e9c8689eb admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:22.717447 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 802/1592] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:44:22.718221 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ca1db-4470-4114-8c97-519e9c8689eb admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:22.743243 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:22.763749 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 791/1593] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:44:22 2022] POST /identity/v3/users => generated 286 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:44:22.764788 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff0be5d-c4bd-40c1-8a7f-f2180534aa4a admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:22.775739 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff0be5d-c4bd-40c1-8a7f-f2180534aa4a admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:22.780934 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0db7dec-108b-418e-bfb1-88ef4c0f9e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:22.781563 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db7dec-108b-418e-bfb1-88ef4c0f9e90 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:22.781841 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db7dec-108b-418e-bfb1-88ef4c0f9e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:22.782178 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db7dec-108b-418e-bfb1-88ef4c0f9e90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:22.782829 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0db7dec-108b-418e-bfb1-88ef4c0f9e90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:22.784519 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0db7dec-108b-418e-bfb1-88ef4c0f9e90 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:22.789935 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ca1db-4470-4114-8c97-519e9c8689eb admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:22.794120 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc69fe86-ba7a-46d2-9cc6-5040320a9c9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:22.794715 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69fe86-ba7a-46d2-9cc6-5040320a9c9b service cinder] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:22.795000 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69fe86-ba7a-46d2-9cc6-5040320a9c9b service cinder] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:22.795276 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69fe86-ba7a-46d2-9cc6-5040320a9c9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:22.796776 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 792/1594] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:44:22.797384 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0db7dec-108b-418e-bfb1-88ef4c0f9e90 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:22.812449 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc69fe86-ba7a-46d2-9cc6-5040320a9c9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:22.813951 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc69fe86-ba7a-46d2-9cc6-5040320a9c9b service cinder] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:22.815339 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0db7dec-108b-418e-bfb1-88ef4c0f9e90 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:22.820789 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a3817-f375-4dc7-a18d-aedba670ef70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:22.821626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a3817-f375-4dc7-a18d-aedba670ef70 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:22.821741 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a3817-f375-4dc7-a18d-aedba670ef70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:22.821967 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a3817-f375-4dc7-a18d-aedba670ef70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:22.822682 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843a3817-f375-4dc7-a18d-aedba670ef70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:22.824513 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843a3817-f375-4dc7-a18d-aedba670ef70 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:22.827217 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 803/1595] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:44:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:44:22.829818 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc69fe86-ba7a-46d2-9cc6-5040320a9c9b service cinder] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:22.839282 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 793/1596] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:44:22.840099 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a3817-f375-4dc7-a18d-aedba670ef70 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:22.850071 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc69fe86-ba7a-46d2-9cc6-5040320a9c9b service cinder] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:22.851196 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a3817-f375-4dc7-a18d-aedba670ef70 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:22.856179 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02195eb2-509e-4b36-823c-d1eea941f174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:22.856841 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02195eb2-509e-4b36-823c-d1eea941f174 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:22.857126 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02195eb2-509e-4b36-823c-d1eea941f174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:22.857404 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02195eb2-509e-4b36-823c-d1eea941f174 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:22.858059 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02195eb2-509e-4b36-823c-d1eea941f174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:22.858736 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9b796-6725-410e-8514-c03add3c633c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:22.859835 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02195eb2-509e-4b36-823c-d1eea941f174 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:22.860188 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9b796-6725-410e-8514-c03add3c633c admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:22.860802 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9b796-6725-410e-8514-c03add3c633c admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:22.861603 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9b796-6725-410e-8514-c03add3c633c admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:22.863190 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b9b796-6725-410e-8514-c03add3c633c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:22.867383 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b9b796-6725-410e-8514-c03add3c633c admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:22.871550 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 794/1597] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:44:22.894727 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02195eb2-509e-4b36-823c-d1eea941f174 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:22.894893 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 804/1598] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:44:22.894893 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9b796-6725-410e-8514-c03add3c633c admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:22.895074 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02195eb2-509e-4b36-823c-d1eea941f174 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:22.895074 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0206a8d8-3d1e-4e1d-8018-04128cdc2cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:22.895074 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206a8d8-3d1e-4e1d-8018-04128cdc2cc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:22.895074 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206a8d8-3d1e-4e1d-8018-04128cdc2cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:22.895451 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206a8d8-3d1e-4e1d-8018-04128cdc2cc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:22.896073 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9b796-6725-410e-8514-c03add3c633c admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:22.896166 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0206a8d8-3d1e-4e1d-8018-04128cdc2cc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:22.896166 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0206a8d8-3d1e-4e1d-8018-04128cdc2cc7 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:22.900390 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d6710-b5c4-4e5d-a64e-377c3a9244b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:22.901070 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d6710-b5c4-4e5d-a64e-377c3a9244b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:22.901365 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d6710-b5c4-4e5d-a64e-377c3a9244b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:22.901632 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d6710-b5c4-4e5d-a64e-377c3a9244b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:22.902297 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d6710-b5c4-4e5d-a64e-377c3a9244b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:22.903916 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d6710-b5c4-4e5d-a64e-377c3a9244b2 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:22.909867 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 795/1599] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:44:22.909867 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0206a8d8-3d1e-4e1d-8018-04128cdc2cc7 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:22.919215 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 805/1600] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:44:22.919215 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d6710-b5c4-4e5d-a64e-377c3a9244b2 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:22.922485 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0206a8d8-3d1e-4e1d-8018-04128cdc2cc7 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:22.928002 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66470eea-f1a8-4283-8ada-f1dfd2d4eae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:22.928652 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66470eea-f1a8-4283-8ada-f1dfd2d4eae0 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:22.928749 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d6710-b5c4-4e5d-a64e-377c3a9244b2 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:22.928906 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66470eea-f1a8-4283-8ada-f1dfd2d4eae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:22.929144 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66470eea-f1a8-4283-8ada-f1dfd2d4eae0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:22.929791 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66470eea-f1a8-4283-8ada-f1dfd2d4eae0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:22.932511 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66470eea-f1a8-4283-8ada-f1dfd2d4eae0 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:22.934607 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e5c5cc-fe68-49a8-93a3-bdc5699e57ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:22.935421 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5c5cc-fe68-49a8-93a3-bdc5699e57ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:22.935833 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5c5cc-fe68-49a8-93a3-bdc5699e57ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:22.936154 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5c5cc-fe68-49a8-93a3-bdc5699e57ad admin admin] PATH_INFO: `/v3/projects/eb377db175354cac93e5ff6579af3e3e/users/9bda9f6f00c74112a54573c3905ad1bf/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:22.945644 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 796/1601] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:44:22.946906 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66470eea-f1a8-4283-8ada-f1dfd2d4eae0 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:22.957808 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e5c5cc-fe68-49a8-93a3-bdc5699e57ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb377db175354cac93e5ff6579af3e3e, user_id=9bda9f6f00c74112a54573c3905ad1bf, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:22.958710 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66470eea-f1a8-4283-8ada-f1dfd2d4eae0 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:22.959762 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: WARNING py.warnings [None req-43e5c5cc-fe68-49a8-93a3-bdc5699e57ad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 05:44:22.959762 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: warnings.warn(msg) Aug 24 05:44:22.959762 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: Aug 24 05:44:22.961887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e5c5cc-fe68-49a8-93a3-bdc5699e57ad admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:22.971057 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f52654-ed84-4310-9dcf-c237fabd86dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:22.975309 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f52654-ed84-4310-9dcf-c237fabd86dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:22.975309 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f52654-ed84-4310-9dcf-c237fabd86dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:22.975309 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f52654-ed84-4310-9dcf-c237fabd86dd admin admin] PATH_INFO: `/v3/projects/9a7253a42718487081b39446c559cd88/users/be0e0097e24e4cc9a16a3066c04104ce/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:22.992839 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f52654-ed84-4310-9dcf-c237fabd86dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a7253a42718487081b39446c559cd88, user_id=be0e0097e24e4cc9a16a3066c04104ce, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:22.995592 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f52654-ed84-4310-9dcf-c237fabd86dd admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:23.094140 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 806/1603] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:44:22 2022] PUT /identity/v3/projects/eb377db175354cac93e5ff6579af3e3e/users/9bda9f6f00c74112a54573c3905ad1bf/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:44:23.095383 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e5c5cc-fe68-49a8-93a3-bdc5699e57ad admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:23.100934 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 797/1603] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:44:22 2022] PUT /identity/v3/projects/9a7253a42718487081b39446c559cd88/users/be0e0097e24e4cc9a16a3066c04104ce/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:44:23.101838 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f52654-ed84-4310-9dcf-c237fabd86dd admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:23.171168 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e5c5cc-fe68-49a8-93a3-bdc5699e57ad admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:23.176439 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d153f8-f1c2-468b-a979-806fb2b99ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:23.179132 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d153f8-f1c2-468b-a979-806fb2b99ca6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:23.179132 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d153f8-f1c2-468b-a979-806fb2b99ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:23.179132 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d153f8-f1c2-468b-a979-806fb2b99ca6 admin admin] PATH_INFO: `/v3/projects/a3ac47b8d1a04316be031edc82d57a7a/users/36e1111855114cd0aa7dce35d38a396f/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:23.179458 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f52654-ed84-4310-9dcf-c237fabd86dd admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:23.186294 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4515cf-89c7-4755-bb73-074f0a3c1924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:23.187295 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4515cf-89c7-4755-bb73-074f0a3c1924 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:23.187295 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4515cf-89c7-4755-bb73-074f0a3c1924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:23.187494 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4515cf-89c7-4755-bb73-074f0a3c1924 admin admin] PATH_INFO: `/v3/projects/cfbd03315ded40279308fcc7f0dde7cb/users/d3e6e20d5cd44444b43b1e7abb240f8d/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:23.198136 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d153f8-f1c2-468b-a979-806fb2b99ca6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3ac47b8d1a04316be031edc82d57a7a, user_id=36e1111855114cd0aa7dce35d38a396f, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:23.200859 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d153f8-f1c2-468b-a979-806fb2b99ca6 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:23.211634 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4515cf-89c7-4755-bb73-074f0a3c1924 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfbd03315ded40279308fcc7f0dde7cb, user_id=d3e6e20d5cd44444b43b1e7abb240f8d, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:23.214475 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4515cf-89c7-4755-bb73-074f0a3c1924 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:23.310745 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 807/1604] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:44:23 2022] PUT /identity/v3/projects/a3ac47b8d1a04316be031edc82d57a7a/users/36e1111855114cd0aa7dce35d38a396f/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:44:23.311598 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d153f8-f1c2-468b-a979-806fb2b99ca6 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:23.378541 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d153f8-f1c2-468b-a979-806fb2b99ca6 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:23.378930 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 798/1605] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:44:23 2022] PUT /identity/v3/projects/cfbd03315ded40279308fcc7f0dde7cb/users/d3e6e20d5cd44444b43b1e7abb240f8d/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:44:23.379796 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4515cf-89c7-4755-bb73-074f0a3c1924 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:23.468791 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4515cf-89c7-4755-bb73-074f0a3c1924 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:23.473363 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f865635c-c05d-4afd-bfbe-235781e44a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:23.474096 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865635c-c05d-4afd-bfbe-235781e44a05 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:23.474363 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865635c-c05d-4afd-bfbe-235781e44a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:23.474503 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c269805-3847-45ab-84e4-b2c90955b2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:23.474665 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865635c-c05d-4afd-bfbe-235781e44a05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:23.475297 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f865635c-c05d-4afd-bfbe-235781e44a05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:23.475399 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c269805-3847-45ab-84e4-b2c90955b2c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:23.475673 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c269805-3847-45ab-84e4-b2c90955b2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:23.476043 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c269805-3847-45ab-84e4-b2c90955b2c9 admin admin] PATH_INFO: `/v3/projects/39a99050c88d48db851f437f68b65a58/users/926115c2e54741e6a3aca02437309b8f/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:23.476964 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f865635c-c05d-4afd-bfbe-235781e44a05 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:23.491370 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 808/1606] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:44:23 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:44:23.496018 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f865635c-c05d-4afd-bfbe-235781e44a05 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:23.496150 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:23.497689 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c269805-3847-45ab-84e4-b2c90955b2c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39a99050c88d48db851f437f68b65a58, user_id=926115c2e54741e6a3aca02437309b8f, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:23.500911 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c269805-3847-45ab-84e4-b2c90955b2c9 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:23.506782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f865635c-c05d-4afd-bfbe-235781e44a05 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:23.511699 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf0a669-6ae3-4e5f-a523-9469698472c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:23.512327 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0a669-6ae3-4e5f-a523-9469698472c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:23.512647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0a669-6ae3-4e5f-a523-9469698472c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:23.512927 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0a669-6ae3-4e5f-a523-9469698472c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:23.513576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf0a669-6ae3-4e5f-a523-9469698472c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:23.515440 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf0a669-6ae3-4e5f-a523-9469698472c1 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:23.527993 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 809/1607] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:44:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:44:23.529350 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f905f71-1337-4eff-873b-02d9e27913a7 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:23.529636 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f905f71-1337-4eff-873b-02d9e27913a7 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:23.529914 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f905f71-1337-4eff-873b-02d9e27913a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:23.550513 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-9f905f71-1337-4eff-873b-02d9e27913a7 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:23.558910 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-9f905f71-1337-4eff-873b-02d9e27913a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:23.560133 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 810/1608] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:23.561364 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d283df-b695-42d8-8916-509efcca1903 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:23.561681 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d283df-b695-42d8-8916-509efcca1903 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:23.561983 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d283df-b695-42d8-8916-509efcca1903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:23.591994 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-d6d283df-b695-42d8-8916-509efcca1903 None None] MFA Rules not processed for user `be0e0097e24e4cc9a16a3066c04104ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:23.593400 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-d6d283df-b695-42d8-8916-509efcca1903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:23.594082 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 799/1609] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:44:23 2022] PUT /identity/v3/projects/39a99050c88d48db851f437f68b65a58/users/926115c2e54741e6a3aca02437309b8f/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:44:23.596126 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cee184-9e9f-4782-9d40-1150d30d7550 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:23.596663 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cee184-9e9f-4782-9d40-1150d30d7550 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:23.597023 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cee184-9e9f-4782-9d40-1150d30d7550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:23.631394 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-a9cee184-9e9f-4782-9d40-1150d30d7550 None None] MFA Rules not processed for user `36e1111855114cd0aa7dce35d38a396f`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:23.632855 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-a9cee184-9e9f-4782-9d40-1150d30d7550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:23.669832 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 811/1610] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:23.671142 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602132ee-0171-450d-a276-693e39263bd1 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:23.671433 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602132ee-0171-450d-a276-693e39263bd1 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:23.671712 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602132ee-0171-450d-a276-693e39263bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:23.677310 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:23.702591 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-602132ee-0171-450d-a276-693e39263bd1 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:23.704894 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-602132ee-0171-450d-a276-693e39263bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:23.726569 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 800/1611] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:23.726569 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cee184-9e9f-4782-9d40-1150d30d7550 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:23.781111 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 812/1612] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:23.782137 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602132ee-0171-450d-a276-693e39263bd1 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:23.791717 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602132ee-0171-450d-a276-693e39263bd1 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:23.797184 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cd475b-c449-4028-96ac-1152bf2b9b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:23.797919 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd475b-c449-4028-96ac-1152bf2b9b1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:23.798224 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd475b-c449-4028-96ac-1152bf2b9b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:23.798505 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd475b-c449-4028-96ac-1152bf2b9b1e admin admin] PATH_INFO: `/v3/projects/e9111027e9024cdc8418e2a2cc39e6d8/users/ad07185f9b8f408891ad37486724d4de/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:23.799724 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cee184-9e9f-4782-9d40-1150d30d7550 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:23.804830 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4967f3c0-e704-4f91-a6f1-7933770c9fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:23.805513 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967f3c0-e704-4f91-a6f1-7933770c9fe8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:23.805798 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967f3c0-e704-4f91-a6f1-7933770c9fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:23.806103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967f3c0-e704-4f91-a6f1-7933770c9fe8 admin admin] PATH_INFO: `/v3/projects/1f6545fe1acc4a73a766dd17cd26678a/users/4638fb875c0e4afbad0695c561db3f53/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:23.819651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cd475b-c449-4028-96ac-1152bf2b9b1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9111027e9024cdc8418e2a2cc39e6d8, user_id=ad07185f9b8f408891ad37486724d4de, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:23.825190 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cd475b-c449-4028-96ac-1152bf2b9b1e admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:23.829831 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4967f3c0-e704-4f91-a6f1-7933770c9fe8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f6545fe1acc4a73a766dd17cd26678a, user_id=4638fb875c0e4afbad0695c561db3f53, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:23.835555 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4967f3c0-e704-4f91-a6f1-7933770c9fe8 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:23.962928 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 813/1613] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:44:23 2022] PUT /identity/v3/projects/e9111027e9024cdc8418e2a2cc39e6d8/users/ad07185f9b8f408891ad37486724d4de/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:44:23.962928 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cd475b-c449-4028-96ac-1152bf2b9b1e admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:23.991872 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 801/1614] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:44:23 2022] PUT /identity/v3/projects/1f6545fe1acc4a73a766dd17cd26678a/users/4638fb875c0e4afbad0695c561db3f53/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:44:23.993451 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f0787-b67c-4ae2-b1d2-618b507b5f74 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:23.993885 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f0787-b67c-4ae2-b1d2-618b507b5f74 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:23.994266 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f0787-b67c-4ae2-b1d2-618b507b5f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:24.033119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-527f0787-b67c-4ae2-b1d2-618b507b5f74 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:24.033119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-527f0787-b67c-4ae2-b1d2-618b507b5f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:24.053950 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cd475b-c449-4028-96ac-1152bf2b9b1e admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:24.063178 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:24.085792 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:24.121799 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 802/1615] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:24.123000 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f0787-b67c-4ae2-b1d2-618b507b5f74 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:24.128143 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce5e9ed-b32d-4641-a11a-a5d408e19614 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:24.129282 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5e9ed-b32d-4641-a11a-a5d408e19614 service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:24.129775 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5e9ed-b32d-4641-a11a-a5d408e19614 service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:24.131845 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5e9ed-b32d-4641-a11a-a5d408e19614 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:24.135261 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f0787-b67c-4ae2-b1d2-618b507b5f74 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:24.143881 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25429210-ed1a-4b63-8693-bdcb92fe0858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:24.144666 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25429210-ed1a-4b63-8693-bdcb92fe0858 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:24.145022 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25429210-ed1a-4b63-8693-bdcb92fe0858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:24.145432 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25429210-ed1a-4b63-8693-bdcb92fe0858 admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:24.145626 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce5e9ed-b32d-4641-a11a-a5d408e19614 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:24.146104 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25429210-ed1a-4b63-8693-bdcb92fe0858 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:24.148021 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce5e9ed-b32d-4641-a11a-a5d408e19614 service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:24.148227 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25429210-ed1a-4b63-8693-bdcb92fe0858 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:24.161220 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 814/1616] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:44:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:44:24.162871 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce5e9ed-b32d-4641-a11a-a5d408e19614 service neutron] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:24.177553 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce5e9ed-b32d-4641-a11a-a5d408e19614 service neutron] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:24.189189 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd13fc80-92e4-43f9-a3d8-c89e45dac9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:24.189999 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13fc80-92e4-43f9-a3d8-c89e45dac9db admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:24.190381 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13fc80-92e4-43f9-a3d8-c89e45dac9db admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:24.190730 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13fc80-92e4-43f9-a3d8-c89e45dac9db admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:24.191586 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd13fc80-92e4-43f9-a3d8-c89e45dac9db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:24.194906 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd13fc80-92e4-43f9-a3d8-c89e45dac9db admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:24.284494 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 803/1617] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:44:24 2022] POST /identity/v3/projects => generated 387 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:44:24.285647 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25429210-ed1a-4b63-8693-bdcb92fe0858 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:24.348158 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25429210-ed1a-4b63-8693-bdcb92fe0858 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:24.352216 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 815/1618] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:44:24 2022] POST /identity/v3/projects => generated 365 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:44:24.352783 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e43fbc-73d8-4843-acfe-ce535a5a5d46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:24.353655 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e43fbc-73d8-4843-acfe-ce535a5a5d46 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:24.353780 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865900ac-64f2-47ff-a8c5-bf5c3d387b0c None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:24.354102 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865900ac-64f2-47ff-a8c5-bf5c3d387b0c None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:24.354353 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865900ac-64f2-47ff-a8c5-bf5c3d387b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:24.354546 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e43fbc-73d8-4843-acfe-ce535a5a5d46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:24.354963 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e43fbc-73d8-4843-acfe-ce535a5a5d46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:24.384194 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-865900ac-64f2-47ff-a8c5-bf5c3d387b0c None None] MFA Rules not processed for user `ad07185f9b8f408891ad37486724d4de`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:24.385512 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-865900ac-64f2-47ff-a8c5-bf5c3d387b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:24.429787 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e43fbc-73d8-4843-acfe-ce535a5a5d46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:24.431252 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e43fbc-73d8-4843-acfe-ce535a5a5d46 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:24.443683 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 804/1619] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:44:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:44:24.445145 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38088068-f42b-4208-ac41-55d1eb5e0dd5 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:24.445496 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38088068-f42b-4208-ac41-55d1eb5e0dd5 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:24.445850 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38088068-f42b-4208-ac41-55d1eb5e0dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:24.479562 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-38088068-f42b-4208-ac41-55d1eb5e0dd5 None None] MFA Rules not processed for user `4638fb875c0e4afbad0695c561db3f53`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:24.480833 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-38088068-f42b-4208-ac41-55d1eb5e0dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:24.485619 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 816/1620] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:24.486484 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865900ac-64f2-47ff-a8c5-bf5c3d387b0c None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:24.507324 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:24.530481 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:24.546138 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865900ac-64f2-47ff-a8c5-bf5c3d387b0c None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:24.550030 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc644a2-632e-432e-8767-78285c0bbc2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:24.550659 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc644a2-632e-432e-8767-78285c0bbc2e service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:24.550947 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc644a2-632e-432e-8767-78285c0bbc2e service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:24.551227 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc644a2-632e-432e-8767-78285c0bbc2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:24.560844 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 805/1621] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:24.561976 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f457e-ae8b-447d-85f6-e34c6f304232 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:24.562268 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f457e-ae8b-447d-85f6-e34c6f304232 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:24.562569 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f457e-ae8b-447d-85f6-e34c6f304232 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:24.563950 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 806/1622] 10.222.0.15 () {54 vars in 962 bytes} [Wed Aug 24 05:44:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:44:24.564540 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f457e-ae8b-447d-85f6-e34c6f304232 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:24.622977 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc644a2-632e-432e-8767-78285c0bbc2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:24.624545 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc644a2-632e-432e-8767-78285c0bbc2e service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:24.636762 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 817/1623] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:44:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:44:24.637694 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc644a2-632e-432e-8767-78285c0bbc2e service neutron] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:24.649241 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f457e-ae8b-447d-85f6-e34c6f304232 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:24.649795 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc644a2-632e-432e-8767-78285c0bbc2e service neutron] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:24.654787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd0029a-0fef-4e02-849e-9e4f8e7d2110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:24.655401 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0029a-0fef-4e02-849e-9e4f8e7d2110 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:24.655682 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0029a-0fef-4e02-849e-9e4f8e7d2110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:24.655954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0029a-0fef-4e02-849e-9e4f8e7d2110 admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:24.656068 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629de528-ba62-4935-9bca-cfb3883161fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:24.656616 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd0029a-0fef-4e02-849e-9e4f8e7d2110 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:24.658550 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd0029a-0fef-4e02-849e-9e4f8e7d2110 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:24.660911 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629de528-ba62-4935-9bca-cfb3883161fa admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:24.662909 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629de528-ba62-4935-9bca-cfb3883161fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:24.663015 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629de528-ba62-4935-9bca-cfb3883161fa admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:24.664230 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629de528-ba62-4935-9bca-cfb3883161fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:24.667110 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629de528-ba62-4935-9bca-cfb3883161fa admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:24.767713 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 818/1624] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:44:24 2022] POST /identity/v3/users => generated 289 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:44:24.767713 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd0029a-0fef-4e02-849e-9e4f8e7d2110 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:24.776962 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd0029a-0fef-4e02-849e-9e4f8e7d2110 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:24.784444 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5601b7-2754-4818-9e0d-9f7f2422bc07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:24.785515 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5601b7-2754-4818-9e0d-9f7f2422bc07 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:24.786084 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5601b7-2754-4818-9e0d-9f7f2422bc07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:24.786598 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5601b7-2754-4818-9e0d-9f7f2422bc07 admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:24.788159 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5601b7-2754-4818-9e0d-9f7f2422bc07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:24.792499 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5601b7-2754-4818-9e0d-9f7f2422bc07 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:24.831777 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 807/1625] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:44:24 2022] POST /identity/v3/users => generated 300 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:44:24.840912 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629de528-ba62-4935-9bca-cfb3883161fa admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:24.965257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629de528-ba62-4935-9bca-cfb3883161fa admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:24.966946 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 819/1626] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:44:24 2022] POST /identity/v3/projects => generated 373 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:44:24.968477 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe0e6e-a6c0-4902-b7a5-fe670b4442f2 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:24.969068 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe0e6e-a6c0-4902-b7a5-fe670b4442f2 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:24.969419 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe0e6e-a6c0-4902-b7a5-fe670b4442f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:25.002753 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-b7fe0e6e-a6c0-4902-b7a5-fe670b4442f2 None None] MFA Rules not processed for user `fbecffb800584c5786813accd061421b`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:25.004450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-b7fe0e6e-a6c0-4902-b7a5-fe670b4442f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:25.082405 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a61a4-bb24-4f13-9f4d-fab5b48ffd63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:25.084900 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a61a4-bb24-4f13-9f4d-fab5b48ffd63 service nova] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:25.085697 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a61a4-bb24-4f13-9f4d-fab5b48ffd63 service nova] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:25.086082 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a61a4-bb24-4f13-9f4d-fab5b48ffd63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:25.093381 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:25.115993 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 820/1627] 10.222.0.15 () {58 vars in 1059 bytes} [Wed Aug 24 05:44:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:25.117191 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fe0e6e-a6c0-4902-b7a5-fe670b4442f2 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:25.149049 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:25.190004 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103a61a4-bb24-4f13-9f4d-fab5b48ffd63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:25.193770 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103a61a4-bb24-4f13-9f4d-fab5b48ffd63 service nova] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:25.220362 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fe0e6e-a6c0-4902-b7a5-fe670b4442f2 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:25.224652 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 808/1628] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:44:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:44:25.226447 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33ad9de-9585-462b-9a03-3b83cc74ce63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:25.227087 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ad9de-9585-462b-9a03-3b83cc74ce63 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:25.227507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ad9de-9585-462b-9a03-3b83cc74ce63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:25.227730 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a61a4-bb24-4f13-9f4d-fab5b48ffd63 service nova] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:25.227898 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ad9de-9585-462b-9a03-3b83cc74ce63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:25.228796 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33ad9de-9585-462b-9a03-3b83cc74ce63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:25.230758 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33ad9de-9585-462b-9a03-3b83cc74ce63 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:25.247622 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 821/1629] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:44:25 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:44:25.248992 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33ad9de-9585-462b-9a03-3b83cc74ce63 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:25.254101 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a61a4-bb24-4f13-9f4d-fab5b48ffd63 service nova] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:25.260008 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e20458-74ef-46e9-9a68-f02b15514b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:25.260873 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e20458-74ef-46e9-9a68-f02b15514b2a admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:25.261589 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e20458-74ef-46e9-9a68-f02b15514b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:25.261994 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e20458-74ef-46e9-9a68-f02b15514b2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:25.263147 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33ad9de-9585-462b-9a03-3b83cc74ce63 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:25.264250 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e20458-74ef-46e9-9a68-f02b15514b2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:25.268215 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e20458-74ef-46e9-9a68-f02b15514b2a admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:25.268660 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4512c35e-1a79-492d-9d96-cc0338f3ffae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:25.269462 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512c35e-1a79-492d-9d96-cc0338f3ffae admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:25.271302 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512c35e-1a79-492d-9d96-cc0338f3ffae admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:25.271302 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512c35e-1a79-492d-9d96-cc0338f3ffae admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:25.271302 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4512c35e-1a79-492d-9d96-cc0338f3ffae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:25.274311 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4512c35e-1a79-492d-9d96-cc0338f3ffae admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:25.288166 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 809/1630] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:44:25 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:44:25.289766 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ebd826-fcb5-4fee-a602-6e4d62557ed7 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:25.291021 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ebd826-fcb5-4fee-a602-6e4d62557ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:25.291021 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ebd826-fcb5-4fee-a602-6e4d62557ed7 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:25.292736 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 810/1631] 10.222.0.15 () {54 vars in 912 bytes} [Wed Aug 24 05:44:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:44:25.293358 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ebd826-fcb5-4fee-a602-6e4d62557ed7 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:25.325266 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ebd826-fcb5-4fee-a602-6e4d62557ed7 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:25.325266 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e368c1-2a3f-4e4f-a5c7-a1af591443db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:25.325266 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e368c1-2a3f-4e4f-a5c7-a1af591443db admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:25.325266 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e368c1-2a3f-4e4f-a5c7-a1af591443db admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:25.325764 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e368c1-2a3f-4e4f-a5c7-a1af591443db admin admin] PATH_INFO: `/v3/projects/ca5768f1027149dfa11b76d8c1196ded/users/471f506156a94b9989f4e228164c0b34/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:25.356686 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e368c1-2a3f-4e4f-a5c7-a1af591443db admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca5768f1027149dfa11b76d8c1196ded, user_id=471f506156a94b9989f4e228164c0b34, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:25.356686 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e368c1-2a3f-4e4f-a5c7-a1af591443db admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:25.390720 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 822/1632] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:44:25 2022] POST /identity/v3/users => generated 293 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:44:25.390720 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4512c35e-1a79-492d-9d96-cc0338f3ffae admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:25.416702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4512c35e-1a79-492d-9d96-cc0338f3ffae admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:25.432176 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f38cbc-931e-4c94-9136-bfd0634f2198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:25.432907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f38cbc-931e-4c94-9136-bfd0634f2198 admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:25.433192 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f38cbc-931e-4c94-9136-bfd0634f2198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:25.434088 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f38cbc-931e-4c94-9136-bfd0634f2198 admin admin] PATH_INFO: `/v3/projects/0acad7f75c144d948aa511bff88d111b/users/2cb4b8f05d274818909ed54546d56aa2/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:25.455583 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f38cbc-931e-4c94-9136-bfd0634f2198 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0acad7f75c144d948aa511bff88d111b, user_id=2cb4b8f05d274818909ed54546d56aa2, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:25.456877 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 811/1633] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:44:25 2022] PUT /identity/v3/projects/ca5768f1027149dfa11b76d8c1196ded/users/471f506156a94b9989f4e228164c0b34/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:44:25.458750 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd0d1c9-0ce1-43b3-a437-faf7c4704935 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:25.458902 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f38cbc-931e-4c94-9136-bfd0634f2198 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:25.459191 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd0d1c9-0ce1-43b3-a437-faf7c4704935 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:25.459555 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd0d1c9-0ce1-43b3-a437-faf7c4704935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:25.480448 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-afd0d1c9-0ce1-43b3-a437-faf7c4704935 None None] MFA Rules not processed for user `fbecffb800584c5786813accd061421b`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:25.481743 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-afd0d1c9-0ce1-43b3-a437-faf7c4704935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:25.569501 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 812/1634] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Aug 24 05:44:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:25.570736 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd0d1c9-0ce1-43b3-a437-faf7c4704935 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:25.631334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 823/1635] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:44:25 2022] PUT /identity/v3/projects/0acad7f75c144d948aa511bff88d111b/users/2cb4b8f05d274818909ed54546d56aa2/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:44:25.632252 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f38cbc-931e-4c94-9136-bfd0634f2198 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:25.655487 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd0d1c9-0ce1-43b3-a437-faf7c4704935 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:25.713588 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:25.748669 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f38cbc-931e-4c94-9136-bfd0634f2198 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:25.759732 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb04b52-dd7a-48b9-a8f9-2a0f584f4ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:25.761329 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb04b52-dd7a-48b9-a8f9-2a0f584f4ecb admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:25.761896 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb04b52-dd7a-48b9-a8f9-2a0f584f4ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:25.762253 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db323854-21d6-4d2f-bb0d-169587d50f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:25.762390 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb04b52-dd7a-48b9-a8f9-2a0f584f4ecb admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:25.763459 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb04b52-dd7a-48b9-a8f9-2a0f584f4ecb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:25.763593 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db323854-21d6-4d2f-bb0d-169587d50f53 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:25.764119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db323854-21d6-4d2f-bb0d-169587d50f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:25.764921 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db323854-21d6-4d2f-bb0d-169587d50f53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:25.766624 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb04b52-dd7a-48b9-a8f9-2a0f584f4ecb admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:25.769325 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db323854-21d6-4d2f-bb0d-169587d50f53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:25.770268 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db323854-21d6-4d2f-bb0d-169587d50f53 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:25.808541 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 813/1637] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:44:25 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:44:25.808767 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 824/1637] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:44:25 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:44:25.808767 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb04b52-dd7a-48b9-a8f9-2a0f584f4ecb admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:25.808972 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db323854-21d6-4d2f-bb0d-169587d50f53 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:25.835994 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb04b52-dd7a-48b9-a8f9-2a0f584f4ecb admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:25.851223 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c08ab3-589a-4395-adf8-621d326c076c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:25.852547 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c08ab3-589a-4395-adf8-621d326c076c admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:25.853187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c08ab3-589a-4395-adf8-621d326c076c admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:25.853808 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c08ab3-589a-4395-adf8-621d326c076c admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:25.855329 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c08ab3-589a-4395-adf8-621d326c076c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:25.859803 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c08ab3-589a-4395-adf8-621d326c076c admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:25.889808 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 825/1638] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:44:25 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:44:25.893005 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c08ab3-589a-4395-adf8-621d326c076c admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:25.916833 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c08ab3-589a-4395-adf8-621d326c076c admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:25.928760 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96fa13c-dbec-4708-9c3d-e6b74cbe0ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:25.930661 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96fa13c-dbec-4708-9c3d-e6b74cbe0ee1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:25.931341 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96fa13c-dbec-4708-9c3d-e6b74cbe0ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:25.931932 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96fa13c-dbec-4708-9c3d-e6b74cbe0ee1 admin admin] PATH_INFO: `/v3/projects/b28537a07c1542a580896fcec2e52fff/users/4e63bd4c90404294a81f04796820f706/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:25.972089 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db323854-21d6-4d2f-bb0d-169587d50f53 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:25.977422 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96fa13c-dbec-4708-9c3d-e6b74cbe0ee1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b28537a07c1542a580896fcec2e52fff, user_id=4e63bd4c90404294a81f04796820f706, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:25.980844 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96fa13c-dbec-4708-9c3d-e6b74cbe0ee1 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:25.987744 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f3efb-fa91-450e-b8c2-6862209585db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:25.989593 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f3efb-fa91-450e-b8c2-6862209585db service cinder] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:25.990224 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f3efb-fa91-450e-b8c2-6862209585db service cinder] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:25.990797 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f3efb-fa91-450e-b8c2-6862209585db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:26.122446 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 826/1639] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:44:25 2022] PUT /identity/v3/projects/b28537a07c1542a580896fcec2e52fff/users/4e63bd4c90404294a81f04796820f706/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:44:26.124070 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96fa13c-dbec-4708-9c3d-e6b74cbe0ee1 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:26.127857 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1f3efb-fa91-450e-b8c2-6862209585db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:26.130317 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1f3efb-fa91-450e-b8c2-6862209585db service cinder] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:26.168561 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 814/1640] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:44:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:44:26.171614 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f3efb-fa91-450e-b8c2-6862209585db service cinder] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:26.204911 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f3efb-fa91-450e-b8c2-6862209585db service cinder] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:26.309506 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:26.393075 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96fa13c-dbec-4708-9c3d-e6b74cbe0ee1 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:26.397538 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d5bab6-87b1-463d-9986-aab91da78793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:26.399111 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5bab6-87b1-463d-9986-aab91da78793 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:26.399594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5bab6-87b1-463d-9986-aab91da78793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:26.400348 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5bab6-87b1-463d-9986-aab91da78793 admin admin] PATH_INFO: `/v3/projects/0acad7f75c144d948aa511bff88d111b/users/2cb4b8f05d274818909ed54546d56aa2/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:26.409729 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d5bab6-87b1-463d-9986-aab91da78793 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0acad7f75c144d948aa511bff88d111b, user_id=2cb4b8f05d274818909ed54546d56aa2, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:26.410521 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b104776-c84a-4e1a-9c43-2846aae4fd90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:26.412334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b104776-c84a-4e1a-9c43-2846aae4fd90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:26.421039 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d5bab6-87b1-463d-9986-aab91da78793 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:26.426202 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b104776-c84a-4e1a-9c43-2846aae4fd90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:26.426202 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b104776-c84a-4e1a-9c43-2846aae4fd90 admin admin] PATH_INFO: `/v3/projects/ca5768f1027149dfa11b76d8c1196ded/users/471f506156a94b9989f4e228164c0b34/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:26.433880 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b104776-c84a-4e1a-9c43-2846aae4fd90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca5768f1027149dfa11b76d8c1196ded, user_id=471f506156a94b9989f4e228164c0b34, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:26.439277 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b104776-c84a-4e1a-9c43-2846aae4fd90 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:26.453869 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 815/1641] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:44:26 2022] PUT /identity/v3/projects/0acad7f75c144d948aa511bff88d111b/users/2cb4b8f05d274818909ed54546d56aa2/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:44:26.453869 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d5bab6-87b1-463d-9986-aab91da78793 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:26.479305 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 827/1642] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:44:26 2022] PUT /identity/v3/projects/ca5768f1027149dfa11b76d8c1196ded/users/471f506156a94b9989f4e228164c0b34/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:44:26.481730 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7fe20f-a7ce-4ff1-bf87-8cd37ccf2df5 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:26.483136 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7fe20f-a7ce-4ff1-bf87-8cd37ccf2df5 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:26.483427 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7fe20f-a7ce-4ff1-bf87-8cd37ccf2df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:26.831790 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:26.836819 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-4a7fe20f-a7ce-4ff1-bf87-8cd37ccf2df5 None None] MFA Rules not processed for user `2cb4b8f05d274818909ed54546d56aa2`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:26.836819 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-4a7fe20f-a7ce-4ff1-bf87-8cd37ccf2df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:26.838971 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d5bab6-87b1-463d-9986-aab91da78793 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:26.838971 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21387764-1ad7-4eec-9112-5272945ad217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:26.838971 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21387764-1ad7-4eec-9112-5272945ad217 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:26.838971 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21387764-1ad7-4eec-9112-5272945ad217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:26.839715 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21387764-1ad7-4eec-9112-5272945ad217 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:26.839715 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21387764-1ad7-4eec-9112-5272945ad217 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:26.839715 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21387764-1ad7-4eec-9112-5272945ad217 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:26.840122 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 828/1643] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:26 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:26.840247 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 816/1644] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:44:26 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:44:26.840305 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9045efe-5910-4668-8a5a-3b201b22127e None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:26.840305 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9045efe-5910-4668-8a5a-3b201b22127e None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:26.840305 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9045efe-5910-4668-8a5a-3b201b22127e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:26.840562 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21387764-1ad7-4eec-9112-5272945ad217 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:26.840562 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21387764-1ad7-4eec-9112-5272945ad217 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:26.840562 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f72c64-1fe0-42ec-b2d1-e7df148a88ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:26.840562 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f72c64-1fe0-42ec-b2d1-e7df148a88ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:26.840779 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f72c64-1fe0-42ec-b2d1-e7df148a88ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:26.840779 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f72c64-1fe0-42ec-b2d1-e7df148a88ea admin admin] PATH_INFO: `/v3/projects/b28537a07c1542a580896fcec2e52fff/users/4e63bd4c90404294a81f04796820f706/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:26.840779 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f72c64-1fe0-42ec-b2d1-e7df148a88ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=b28537a07c1542a580896fcec2e52fff, user_id=4e63bd4c90404294a81f04796820f706, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:26.840779 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f72c64-1fe0-42ec-b2d1-e7df148a88ea admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:26.842687 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-a9045efe-5910-4668-8a5a-3b201b22127e None None] MFA Rules not processed for user `471f506156a94b9989f4e228164c0b34`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:26.842687 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-a9045efe-5910-4668-8a5a-3b201b22127e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:26.843228 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 817/1645] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:44:26 2022] PUT /identity/v3/projects/b28537a07c1542a580896fcec2e52fff/users/4e63bd4c90404294a81f04796820f706/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:44:26.843228 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f72c64-1fe0-42ec-b2d1-e7df148a88ea admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:26.843228 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f72c64-1fe0-42ec-b2d1-e7df148a88ea admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:26.843228 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01a4975-c792-444c-99e4-ae03da50ca7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:26.843766 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01a4975-c792-444c-99e4-ae03da50ca7b service cinder] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:26.843766 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01a4975-c792-444c-99e4-ae03da50ca7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:26.843766 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01a4975-c792-444c-99e4-ae03da50ca7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:26.844818 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 829/1646] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:26 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:26.844818 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77a400-0a2a-4388-b6d3-0eff06f3bc7b None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:26.844818 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77a400-0a2a-4388-b6d3-0eff06f3bc7b None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:26.844818 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77a400-0a2a-4388-b6d3-0eff06f3bc7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:26.885022 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-de77a400-0a2a-4388-b6d3-0eff06f3bc7b None None] MFA Rules not processed for user `4e63bd4c90404294a81f04796820f706`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:26.886249 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-de77a400-0a2a-4388-b6d3-0eff06f3bc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:26.904631 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01a4975-c792-444c-99e4-ae03da50ca7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:26.906070 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01a4975-c792-444c-99e4-ae03da50ca7b service cinder] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:26.916099 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 818/1647] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:44:26 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:44:26.917139 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01a4975-c792-444c-99e4-ae03da50ca7b service cinder] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:26.925905 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01a4975-c792-444c-99e4-ae03da50ca7b service cinder] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:26.930391 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b48b2a-a164-4b79-adfe-6529c349db32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:26.930998 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b48b2a-a164-4b79-adfe-6529c349db32 service cinder] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:26.931282 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b48b2a-a164-4b79-adfe-6529c349db32 service cinder] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:26.931549 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b48b2a-a164-4b79-adfe-6529c349db32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:26.944415 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b48b2a-a164-4b79-adfe-6529c349db32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:26.944415 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b48b2a-a164-4b79-adfe-6529c349db32 service cinder] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:26.957975 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 819/1648] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:44:26 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:44:26.964978 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b48b2a-a164-4b79-adfe-6529c349db32 service cinder] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:27.011575 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 830/1649] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:26 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:27.030858 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:27.044366 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77a400-0a2a-4388-b6d3-0eff06f3bc7b None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:27.060801 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77a400-0a2a-4388-b6d3-0eff06f3bc7b None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:27.071499 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4273cb0-ec5f-4c81-a63e-b6cea07028c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:27.072146 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4273cb0-ec5f-4c81-a63e-b6cea07028c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:27.072618 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4273cb0-ec5f-4c81-a63e-b6cea07028c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:27.072943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4273cb0-ec5f-4c81-a63e-b6cea07028c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:27.094308 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4273cb0-ec5f-4c81-a63e-b6cea07028c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:27.096662 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4273cb0-ec5f-4c81-a63e-b6cea07028c0 service cinder] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:27.110972 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 831/1650] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:44:27 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:44:27.129211 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b48b2a-a164-4b79-adfe-6529c349db32 service cinder] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:27.135810 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01e9aec-38d2-43d0-9cf9-2c2c2e40531b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:27.140131 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e9aec-38d2-43d0-9cf9-2c2c2e40531b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:27.140131 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e9aec-38d2-43d0-9cf9-2c2c2e40531b admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:27.140131 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e9aec-38d2-43d0-9cf9-2c2c2e40531b admin admin] PATH_INFO: `/v3/users/be0e0097e24e4cc9a16a3066c04104ce` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:27.140131 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01e9aec-38d2-43d0-9cf9-2c2c2e40531b admin admin] RBAC: Authorizing `identity:delete_user(user_id=be0e0097e24e4cc9a16a3066c04104ce)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:27.140697 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01e9aec-38d2-43d0-9cf9-2c2c2e40531b admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:27.520464 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4273cb0-ec5f-4c81-a63e-b6cea07028c0 service cinder] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:27.531271 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4273cb0-ec5f-4c81-a63e-b6cea07028c0 service cinder] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:27.540937 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3b4a7-2775-4355-9a53-b18c40260b84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:27.543322 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3b4a7-2775-4355-9a53-b18c40260b84 service cinder] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:27.543612 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3b4a7-2775-4355-9a53-b18c40260b84 service cinder] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:27.544707 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3b4a7-2775-4355-9a53-b18c40260b84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:27.633095 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-c01e9aec-38d2-43d0-9cf9-2c2c2e40531b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be0e0097e24e4cc9a16a3066c04104ce'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:27.655911 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-c01e9aec-38d2-43d0-9cf9-2c2c2e40531b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be0e0097e24e4cc9a16a3066c04104ce'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:27.661954 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-c01e9aec-38d2-43d0-9cf9-2c2c2e40531b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be0e0097e24e4cc9a16a3066c04104ce'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:27.683733 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa3b4a7-2775-4355-9a53-b18c40260b84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:27.685991 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa3b4a7-2775-4355-9a53-b18c40260b84 service cinder] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:27.702123 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 832/1651] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:44:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:44:27.703630 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3b4a7-2775-4355-9a53-b18c40260b84 service cinder] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:27.715546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3b4a7-2775-4355-9a53-b18c40260b84 service cinder] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:27.719498 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:27.732467 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206fa92-d054-434f-9643-272a859609d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:27.732467 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206fa92-d054-434f-9643-272a859609d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:27.732467 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206fa92-d054-434f-9643-272a859609d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:27.732467 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206fa92-d054-434f-9643-272a859609d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:27.733931 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-c206fa92-d054-434f-9643-272a859609d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:27.813913 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-c01e9aec-38d2-43d0-9cf9-2c2c2e40531b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be0e0097e24e4cc9a16a3066c04104ce'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:27.835002 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c206fa92-d054-434f-9643-272a859609d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:27.836687 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c206fa92-d054-434f-9643-272a859609d6 service cinder] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:27.854832 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-c01e9aec-38d2-43d0-9cf9-2c2c2e40531b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be0e0097e24e4cc9a16a3066c04104ce'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:27.867933 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 833/1652] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:44:27 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:44:27.869010 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 820/1653] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:44:26 2022] DELETE /identity/v3/users/be0e0097e24e4cc9a16a3066c04104ce => generated 0 bytes in 905 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:44:27.873526 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d88fd-12c5-4b87-b7a9-6783538dbd54 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:27.873971 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d88fd-12c5-4b87-b7a9-6783538dbd54 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:27.874264 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d88fd-12c5-4b87-b7a9-6783538dbd54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:27.901283 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-dd1d88fd-12c5-4b87-b7a9-6783538dbd54 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:27.902965 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-dd1d88fd-12c5-4b87-b7a9-6783538dbd54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:28.061082 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 834/1654] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:28.092641 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01e9aec-38d2-43d0-9cf9-2c2c2e40531b admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:28.094271 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-c01e9aec-38d2-43d0-9cf9-2c2c2e40531b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:28.243497 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01e9aec-38d2-43d0-9cf9-2c2c2e40531b admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:28.248555 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275cbec9-0cad-4ebc-9f45-a93ff47fd18c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:28.249433 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275cbec9-0cad-4ebc-9f45-a93ff47fd18c service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:28.249860 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275cbec9-0cad-4ebc-9f45-a93ff47fd18c service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:28.250291 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275cbec9-0cad-4ebc-9f45-a93ff47fd18c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:28.264313 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275cbec9-0cad-4ebc-9f45-a93ff47fd18c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:28.266083 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275cbec9-0cad-4ebc-9f45-a93ff47fd18c service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:28.298379 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 821/1655] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:44:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:44:28.776243 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:29.283851 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:29.319406 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1d88fd-12c5-4b87-b7a9-6783538dbd54 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:29.319406 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-dd1d88fd-12c5-4b87-b7a9-6783538dbd54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:29.346425 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1d88fd-12c5-4b87-b7a9-6783538dbd54 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:29.360341 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2558d0-ad72-4add-a75d-5a50695f6cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:29.361127 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2558d0-ad72-4add-a75d-5a50695f6cc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:29.361420 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2558d0-ad72-4add-a75d-5a50695f6cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:29.361664 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2558d0-ad72-4add-a75d-5a50695f6cc2 admin admin] PATH_INFO: `/v3/users/e9cd9823dd8b4195b6f782b8cf391a07` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:29.363171 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2558d0-ad72-4add-a75d-5a50695f6cc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9cd9823dd8b4195b6f782b8cf391a07)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:29.364995 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: WARNING py.warnings [None req-cc2558d0-ad72-4add-a75d-5a50695f6cc2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 05:44:29.364995 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: warnings.warn(msg) Aug 24 05:44:29.364995 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: Aug 24 05:44:29.366672 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2558d0-ad72-4add-a75d-5a50695f6cc2 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:29.737457 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-cc2558d0-ad72-4add-a75d-5a50695f6cc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9cd9823dd8b4195b6f782b8cf391a07'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:29.808721 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-cc2558d0-ad72-4add-a75d-5a50695f6cc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9cd9823dd8b4195b6f782b8cf391a07'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:29.838177 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-cc2558d0-ad72-4add-a75d-5a50695f6cc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9cd9823dd8b4195b6f782b8cf391a07'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:29.893864 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-cc2558d0-ad72-4add-a75d-5a50695f6cc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9cd9823dd8b4195b6f782b8cf391a07'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:29.900188 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-cc2558d0-ad72-4add-a75d-5a50695f6cc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9cd9823dd8b4195b6f782b8cf391a07'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:29.916282 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 835/1656] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:44:29 2022] DELETE /identity/v3/users/e9cd9823dd8b4195b6f782b8cf391a07 => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:44:29.923300 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d676d1-eeb1-4c84-835f-e94cf3284f7c None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:29.923996 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d676d1-eeb1-4c84-835f-e94cf3284f7c None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:29.923996 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d676d1-eeb1-4c84-835f-e94cf3284f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:29.979169 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-55d676d1-eeb1-4c84-835f-e94cf3284f7c None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:29.995243 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-55d676d1-eeb1-4c84-835f-e94cf3284f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:30.214066 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 822/1657] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:30.221379 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2558d0-ad72-4add-a75d-5a50695f6cc2 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:30.222868 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-cc2558d0-ad72-4add-a75d-5a50695f6cc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:30.368964 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2558d0-ad72-4add-a75d-5a50695f6cc2 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:30.371849 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcd841b-1409-4789-9fb9-dccc5fee45d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:30.373484 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd841b-1409-4789-9fb9-dccc5fee45d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:30.373484 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd841b-1409-4789-9fb9-dccc5fee45d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:30.374066 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd841b-1409-4789-9fb9-dccc5fee45d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:30.402916 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcd841b-1409-4789-9fb9-dccc5fee45d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:30.402916 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcd841b-1409-4789-9fb9-dccc5fee45d4 service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:30.416453 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 836/1658] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:44:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:44:30.605244 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d676d1-eeb1-4c84-835f-e94cf3284f7c None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:30.611690 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-55d676d1-eeb1-4c84-835f-e94cf3284f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:30.639614 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d676d1-eeb1-4c84-835f-e94cf3284f7c None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:30.651570 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edad492-487b-4d3d-8eaa-ea316051cc6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:30.652295 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edad492-487b-4d3d-8eaa-ea316051cc6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:30.652624 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edad492-487b-4d3d-8eaa-ea316051cc6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:30.652910 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edad492-487b-4d3d-8eaa-ea316051cc6c admin admin] PATH_INFO: `/v3/projects/9a7253a42718487081b39446c559cd88` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:30.661415 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edad492-487b-4d3d-8eaa-ea316051cc6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a7253a42718487081b39446c559cd88)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:30.662489 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: WARNING py.warnings [None req-4edad492-487b-4d3d-8eaa-ea316051cc6c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 05:44:30.662489 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: warnings.warn(msg) Aug 24 05:44:30.662489 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: Aug 24 05:44:30.664871 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edad492-487b-4d3d-8eaa-ea316051cc6c admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:30.982445 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-4edad492-487b-4d3d-8eaa-ea316051cc6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a7253a42718487081b39446c559cd88', 'request_id': 'req-4edad492-487b-4d3d-8eaa-ea316051cc6c'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:31.012438 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-4edad492-487b-4d3d-8eaa-ea316051cc6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a7253a42718487081b39446c559cd88', 'request_id': 'req-4edad492-487b-4d3d-8eaa-ea316051cc6c'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:31.110441 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-4edad492-487b-4d3d-8eaa-ea316051cc6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a7253a42718487081b39446c559cd88', 'request_id': 'req-4edad492-487b-4d3d-8eaa-ea316051cc6c'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:31.138492 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-4edad492-487b-4d3d-8eaa-ea316051cc6c admin admin] The token cache is being invalidate because project 9a7253a42718487081b39446c559cd88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:44:31.138798 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-4edad492-487b-4d3d-8eaa-ea316051cc6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:31.140634 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-4edad492-487b-4d3d-8eaa-ea316051cc6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:31.145959 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 823/1659] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:44:30 2022] DELETE /identity/v3/projects/9a7253a42718487081b39446c559cd88 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:44:31.149599 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcd841b-1409-4789-9fb9-dccc5fee45d4 service neutron] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:31.188446 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcd841b-1409-4789-9fb9-dccc5fee45d4 service neutron] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:31.392904 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b6921a-e554-4270-ae86-23680ca61717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:31.394271 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b6921a-e554-4270-ae86-23680ca61717 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:31.394862 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b6921a-e554-4270-ae86-23680ca61717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:31.395488 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b6921a-e554-4270-ae86-23680ca61717 admin admin] PATH_INFO: `/v3/users/2cb4b8f05d274818909ed54546d56aa2` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:31.398479 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b6921a-e554-4270-ae86-23680ca61717 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cb4b8f05d274818909ed54546d56aa2)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:31.402922 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b6921a-e554-4270-ae86-23680ca61717 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:31.422924 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:31.925469 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-28b6921a-e554-4270-ae86-23680ca61717 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cb4b8f05d274818909ed54546d56aa2'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:32.107096 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-28b6921a-e554-4270-ae86-23680ca61717 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cb4b8f05d274818909ed54546d56aa2'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:32.180857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-28b6921a-e554-4270-ae86-23680ca61717 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cb4b8f05d274818909ed54546d56aa2'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:32.234272 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-28b6921a-e554-4270-ae86-23680ca61717 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cb4b8f05d274818909ed54546d56aa2'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:32.236083 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-28b6921a-e554-4270-ae86-23680ca61717 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cb4b8f05d274818909ed54546d56aa2'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:32.265925 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 837/1660] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:44:31 2022] DELETE /identity/v3/users/2cb4b8f05d274818909ed54546d56aa2 => generated 0 bytes in 1118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:44:33.289541 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edad492-487b-4d3d-8eaa-ea316051cc6c admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:33.293506 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-4edad492-487b-4d3d-8eaa-ea316051cc6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:33.319248 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edad492-487b-4d3d-8eaa-ea316051cc6c admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:33.417158 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:33.437674 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec20246-4ad2-4684-91c2-a3c47e28e83d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:33.438349 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec20246-4ad2-4684-91c2-a3c47e28e83d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:33.438639 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec20246-4ad2-4684-91c2-a3c47e28e83d admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:33.438920 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec20246-4ad2-4684-91c2-a3c47e28e83d admin admin] PATH_INFO: `/v3/projects/69d11893b9624f6bbf8c8eeda08d4858` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:33.443038 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec20246-4ad2-4684-91c2-a3c47e28e83d admin admin] RBAC: Authorizing `identity:delete_project(project_id=69d11893b9624f6bbf8c8eeda08d4858)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:33.445240 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec20246-4ad2-4684-91c2-a3c47e28e83d admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:33.706955 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-aec20246-4ad2-4684-91c2-a3c47e28e83d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69d11893b9624f6bbf8c8eeda08d4858', 'request_id': 'req-aec20246-4ad2-4684-91c2-a3c47e28e83d'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:33.724352 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-aec20246-4ad2-4684-91c2-a3c47e28e83d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69d11893b9624f6bbf8c8eeda08d4858', 'request_id': 'req-aec20246-4ad2-4684-91c2-a3c47e28e83d'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:33.820014 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-aec20246-4ad2-4684-91c2-a3c47e28e83d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69d11893b9624f6bbf8c8eeda08d4858', 'request_id': 'req-aec20246-4ad2-4684-91c2-a3c47e28e83d'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:33.844467 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-aec20246-4ad2-4684-91c2-a3c47e28e83d admin admin] The token cache is being invalidate because project 69d11893b9624f6bbf8c8eeda08d4858 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:44:33.844467 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-aec20246-4ad2-4684-91c2-a3c47e28e83d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:33.844467 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-aec20246-4ad2-4684-91c2-a3c47e28e83d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:33.848326 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 824/1661] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:44:33 2022] DELETE /identity/v3/projects/69d11893b9624f6bbf8c8eeda08d4858 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:44:33.853428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7254f97-815e-4a25-a35b-477d52472b46 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:33.854278 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7254f97-815e-4a25-a35b-477d52472b46 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:33.854607 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7254f97-815e-4a25-a35b-477d52472b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:33.884437 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-a7254f97-815e-4a25-a35b-477d52472b46 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:33.885863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-a7254f97-815e-4a25-a35b-477d52472b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:34.019654 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 838/1662] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:34.029357 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237090f9-9931-41e9-8eb8-94a67a07cb00 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:34.030436 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237090f9-9931-41e9-8eb8-94a67a07cb00 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:34.031418 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237090f9-9931-41e9-8eb8-94a67a07cb00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:34.067028 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-237090f9-9931-41e9-8eb8-94a67a07cb00 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:34.070990 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-237090f9-9931-41e9-8eb8-94a67a07cb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:34.088945 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 825/1663] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:34.093477 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7254f97-815e-4a25-a35b-477d52472b46 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:34.112973 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7254f97-815e-4a25-a35b-477d52472b46 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:34.126441 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5970b6a0-e7c5-4b40-b5c8-f8ceea47863b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:34.127486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970b6a0-e7c5-4b40-b5c8-f8ceea47863b admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:34.127987 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970b6a0-e7c5-4b40-b5c8-f8ceea47863b admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:34.129428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970b6a0-e7c5-4b40-b5c8-f8ceea47863b admin admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:34.132757 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5970b6a0-e7c5-4b40-b5c8-f8ceea47863b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:44:34.133546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5970b6a0-e7c5-4b40-b5c8-f8ceea47863b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:34.136585 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5970b6a0-e7c5-4b40-b5c8-f8ceea47863b admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:34.157102 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 839/1664] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:44:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:44:34.161116 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237090f9-9931-41e9-8eb8-94a67a07cb00 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:34.177111 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237090f9-9931-41e9-8eb8-94a67a07cb00 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:34.194363 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea5e3b-2891-4624-a64c-6e8928071d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:34.195062 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea5e3b-2891-4624-a64c-6e8928071d70 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:34.195386 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea5e3b-2891-4624-a64c-6e8928071d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:34.196450 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea5e3b-2891-4624-a64c-6e8928071d70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:34.196450 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eea5e3b-2891-4624-a64c-6e8928071d70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:34.201181 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eea5e3b-2891-4624-a64c-6e8928071d70 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:34.335408 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 826/1665] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:44:34 2022] POST /identity/v3/projects => generated 367 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:44:34.342813 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5970b6a0-e7c5-4b40-b5c8-f8ceea47863b admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:34.371768 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5970b6a0-e7c5-4b40-b5c8-f8ceea47863b admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:34.551153 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a4a0d-f838-444c-bb7f-23364359d5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:34.552785 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a4a0d-f838-444c-bb7f-23364359d5f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:34.552785 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a4a0d-f838-444c-bb7f-23364359d5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:34.552785 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a4a0d-f838-444c-bb7f-23364359d5f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:34.554575 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a4a0d-f838-444c-bb7f-23364359d5f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:34.557247 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a4a0d-f838-444c-bb7f-23364359d5f1 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:34.641201 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea5e3b-2891-4624-a64c-6e8928071d70 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:34.643397 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-9eea5e3b-2891-4624-a64c-6e8928071d70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:34.681450 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea5e3b-2891-4624-a64c-6e8928071d70 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:34.716049 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1097b124-2bb7-4c94-9652-e0dce05e8805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:34.717035 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097b124-2bb7-4c94-9652-e0dce05e8805 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:34.717512 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097b124-2bb7-4c94-9652-e0dce05e8805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:34.718251 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097b124-2bb7-4c94-9652-e0dce05e8805 admin admin] PATH_INFO: `/v3/projects/0acad7f75c144d948aa511bff88d111b` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:34.722376 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1097b124-2bb7-4c94-9652-e0dce05e8805 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0acad7f75c144d948aa511bff88d111b)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:34.724388 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1097b124-2bb7-4c94-9652-e0dce05e8805 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:34.771719 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:34.771719 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 840/1666] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:44:34 2022] POST /identity/v3/users => generated 284 bytes in 431 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:44:34.775306 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a4a0d-f838-444c-bb7f-23364359d5f1 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:34.822746 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a4a0d-f838-444c-bb7f-23364359d5f1 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:34.827976 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bacfa8-229e-4d34-95bb-9f62c51f4daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:34.829126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bacfa8-229e-4d34-95bb-9f62c51f4daf admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:34.829610 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bacfa8-229e-4d34-95bb-9f62c51f4daf admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:34.831237 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bacfa8-229e-4d34-95bb-9f62c51f4daf admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:34.834189 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bacfa8-229e-4d34-95bb-9f62c51f4daf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:34.835983 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bacfa8-229e-4d34-95bb-9f62c51f4daf admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:34.868009 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 841/1667] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:44:34.875881 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bacfa8-229e-4d34-95bb-9f62c51f4daf admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:34.898801 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bacfa8-229e-4d34-95bb-9f62c51f4daf admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:34.908295 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ed939-66de-4e88-8457-39067cdf78b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:34.908295 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ed939-66de-4e88-8457-39067cdf78b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:34.908295 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ed939-66de-4e88-8457-39067cdf78b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:34.908295 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ed939-66de-4e88-8457-39067cdf78b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:34.908676 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ed939-66de-4e88-8457-39067cdf78b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:34.909973 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ed939-66de-4e88-8457-39067cdf78b7 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:34.929797 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 842/1668] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:44:34.943045 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ed939-66de-4e88-8457-39067cdf78b7 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:34.990749 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-1097b124-2bb7-4c94-9652-e0dce05e8805 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0acad7f75c144d948aa511bff88d111b', 'request_id': 'req-1097b124-2bb7-4c94-9652-e0dce05e8805'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:35.000844 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-1097b124-2bb7-4c94-9652-e0dce05e8805 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0acad7f75c144d948aa511bff88d111b', 'request_id': 'req-1097b124-2bb7-4c94-9652-e0dce05e8805'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:35.008150 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ed939-66de-4e88-8457-39067cdf78b7 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:35.172700 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-1097b124-2bb7-4c94-9652-e0dce05e8805 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0acad7f75c144d948aa511bff88d111b', 'request_id': 'req-1097b124-2bb7-4c94-9652-e0dce05e8805'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:35.192464 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-1097b124-2bb7-4c94-9652-e0dce05e8805 admin admin] The token cache is being invalidate because project 0acad7f75c144d948aa511bff88d111b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:44:35.193858 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-1097b124-2bb7-4c94-9652-e0dce05e8805 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:35.196655 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-1097b124-2bb7-4c94-9652-e0dce05e8805 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:35.205919 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 827/1669] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:44:34 2022] DELETE /identity/v3/projects/0acad7f75c144d948aa511bff88d111b => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:44:35.206976 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e15fa7b-63eb-4c65-aa48-4f279c0334dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:35.209273 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15fa7b-63eb-4c65-aa48-4f279c0334dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:35.209806 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15fa7b-63eb-4c65-aa48-4f279c0334dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:35.211656 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be179d7e-c0fd-48db-af78-fd6b2fa5e5e4 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:35.212121 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be179d7e-c0fd-48db-af78-fd6b2fa5e5e4 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:35.212567 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be179d7e-c0fd-48db-af78-fd6b2fa5e5e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:35.214946 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15fa7b-63eb-4c65-aa48-4f279c0334dc admin admin] PATH_INFO: `/v3/projects/cde6e2a745e542619f2ee4cde56f0e8a/users/2facc43582f9418baf93abbedee45f44/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:35.267243 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e15fa7b-63eb-4c65-aa48-4f279c0334dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=cde6e2a745e542619f2ee4cde56f0e8a, user_id=2facc43582f9418baf93abbedee45f44, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:35.284003 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e15fa7b-63eb-4c65-aa48-4f279c0334dc admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:35.285349 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-be179d7e-c0fd-48db-af78-fd6b2fa5e5e4 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:35.288534 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-be179d7e-c0fd-48db-af78-fd6b2fa5e5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:35.320969 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 828/1670] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:44:35.331746 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6cc438-73c2-4125-b35c-677122bca655 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:35.332268 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6cc438-73c2-4125-b35c-677122bca655 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:35.333218 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6cc438-73c2-4125-b35c-677122bca655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:35.397685 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-4d6cc438-73c2-4125-b35c-677122bca655 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:35.398960 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-4d6cc438-73c2-4125-b35c-677122bca655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:35.421109 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 843/1671] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:44:34 2022] PUT /identity/v3/projects/cde6e2a745e542619f2ee4cde56f0e8a/users/2facc43582f9418baf93abbedee45f44/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:44:35.422856 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f3fd67-1eaf-4788-8968-dd7411088f88 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:35.423150 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f3fd67-1eaf-4788-8968-dd7411088f88 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:35.423778 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f3fd67-1eaf-4788-8968-dd7411088f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:35.424697 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 829/1672] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:35.428198 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6cc438-73c2-4125-b35c-677122bca655 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:35.479361 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6cc438-73c2-4125-b35c-677122bca655 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:35.505221 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-47f3fd67-1eaf-4788-8968-dd7411088f88 None None] MFA Rules not processed for user `2facc43582f9418baf93abbedee45f44`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:35.506992 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-47f3fd67-1eaf-4788-8968-dd7411088f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:35.618600 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 844/1673] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:35.627932 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f3fd67-1eaf-4788-8968-dd7411088f88 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:35.632194 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-47f3fd67-1eaf-4788-8968-dd7411088f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:35.745731 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:35.748567 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e142a71-6093-4b42-9c0f-9ef4a5bfba02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:35.750981 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e142a71-6093-4b42-9c0f-9ef4a5bfba02 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:35.754889 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f3fd67-1eaf-4788-8968-dd7411088f88 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:35.758289 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e142a71-6093-4b42-9c0f-9ef4a5bfba02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:35.760832 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e142a71-6093-4b42-9c0f-9ef4a5bfba02 admin admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:35.762109 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e142a71-6093-4b42-9c0f-9ef4a5bfba02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:44:35.762811 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cecd7d3-b41d-4b51-b4e0-731f5119c46e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:35.763371 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e142a71-6093-4b42-9c0f-9ef4a5bfba02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:35.763533 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cecd7d3-b41d-4b51-b4e0-731f5119c46e service cinder] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:35.763777 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cecd7d3-b41d-4b51-b4e0-731f5119c46e service cinder] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:35.764024 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cecd7d3-b41d-4b51-b4e0-731f5119c46e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:35.770198 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e142a71-6093-4b42-9c0f-9ef4a5bfba02 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:35.790677 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 830/1674] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:44:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:44:35.793917 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cecd7d3-b41d-4b51-b4e0-731f5119c46e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:35.795208 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e142a71-6093-4b42-9c0f-9ef4a5bfba02 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:35.795421 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cecd7d3-b41d-4b51-b4e0-731f5119c46e service cinder] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:35.806891 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 845/1675] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:44:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:44:35.814994 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e142a71-6093-4b42-9c0f-9ef4a5bfba02 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:35.820628 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ffeda-2c07-4f20-83b5-22667b0269c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:35.837629 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ffeda-2c07-4f20-83b5-22667b0269c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:35.837629 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ffeda-2c07-4f20-83b5-22667b0269c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:35.837629 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ffeda-2c07-4f20-83b5-22667b0269c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:35.837629 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24ffeda-2c07-4f20-83b5-22667b0269c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:35.837629 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24ffeda-2c07-4f20-83b5-22667b0269c0 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:36.035993 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 831/1676] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:44:35 2022] POST /identity/v3/projects => generated 387 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:44:36.041174 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cecd7d3-b41d-4b51-b4e0-731f5119c46e service cinder] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:36.054244 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cecd7d3-b41d-4b51-b4e0-731f5119c46e service cinder] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:36.150112 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff5cc61-904b-421a-a5e4-469767ed8bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:36.150672 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5cc61-904b-421a-a5e4-469767ed8bff admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:36.150961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5cc61-904b-421a-a5e4-469767ed8bff admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:36.151311 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5cc61-904b-421a-a5e4-469767ed8bff admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:36.152241 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff5cc61-904b-421a-a5e4-469767ed8bff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:36.155574 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff5cc61-904b-421a-a5e4-469767ed8bff admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:36.299846 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 846/1677] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:44:36 2022] POST /identity/v3/users => generated 294 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:44:36.301925 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ffeda-2c07-4f20-83b5-22667b0269c0 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:36.323165 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ffeda-2c07-4f20-83b5-22667b0269c0 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:36.335080 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fe8a34-cf18-461f-be00-4e6606c2daab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:36.335866 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe8a34-cf18-461f-be00-4e6606c2daab admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:36.336374 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe8a34-cf18-461f-be00-4e6606c2daab admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:36.336888 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe8a34-cf18-461f-be00-4e6606c2daab admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:36.337799 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fe8a34-cf18-461f-be00-4e6606c2daab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:36.346019 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fe8a34-cf18-461f-be00-4e6606c2daab admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:36.382811 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 832/1678] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:44:36.387993 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff5cc61-904b-421a-a5e4-469767ed8bff admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:36.405340 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff5cc61-904b-421a-a5e4-469767ed8bff admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:36.428103 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a69bdba-5294-4c14-9ff6-4d738e59b9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:36.429271 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a69bdba-5294-4c14-9ff6-4d738e59b9c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:36.429561 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a69bdba-5294-4c14-9ff6-4d738e59b9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:36.429846 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a69bdba-5294-4c14-9ff6-4d738e59b9c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:36.430516 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a69bdba-5294-4c14-9ff6-4d738e59b9c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:36.432141 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a69bdba-5294-4c14-9ff6-4d738e59b9c9 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:36.459004 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 847/1679] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:44:36.462917 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fe8a34-cf18-461f-be00-4e6606c2daab admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:36.480635 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fe8a34-cf18-461f-be00-4e6606c2daab admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:36.499090 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a94761d-5219-4d9d-8086-2bffd2f97944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:36.501602 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a94761d-5219-4d9d-8086-2bffd2f97944 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:36.502486 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a94761d-5219-4d9d-8086-2bffd2f97944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:36.505231 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a94761d-5219-4d9d-8086-2bffd2f97944 admin admin] PATH_INFO: `/v3/projects/937bc794db7940d0812728d3908701ed/users/9efb2e94f7324d56b1a2cee06d2e9267/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:36.555873 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a94761d-5219-4d9d-8086-2bffd2f97944 admin admin] RBAC: Authorizing `identity:create_grant(project_id=937bc794db7940d0812728d3908701ed, user_id=9efb2e94f7324d56b1a2cee06d2e9267, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:36.561651 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a94761d-5219-4d9d-8086-2bffd2f97944 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:36.697503 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 833/1680] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:44:36 2022] PUT /identity/v3/projects/937bc794db7940d0812728d3908701ed/users/9efb2e94f7324d56b1a2cee06d2e9267/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:44:36.701996 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761f087-8373-4e58-a64e-6cafa39c56f7 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:36.702380 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761f087-8373-4e58-a64e-6cafa39c56f7 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:36.702890 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761f087-8373-4e58-a64e-6cafa39c56f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:36.777463 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-7761f087-8373-4e58-a64e-6cafa39c56f7 None None] MFA Rules not processed for user `9efb2e94f7324d56b1a2cee06d2e9267`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:36.778862 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-7761f087-8373-4e58-a64e-6cafa39c56f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:36.782216 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0157589-481c-4c61-972b-b1f1c4c257bd None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:36.782505 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0157589-481c-4c61-972b-b1f1c4c257bd None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:36.782842 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0157589-481c-4c61-972b-b1f1c4c257bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:36.858067 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-c0157589-481c-4c61-972b-b1f1c4c257bd None None] MFA Rules not processed for user `d3e6e20d5cd44444b43b1e7abb240f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:36.861710 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-c0157589-481c-4c61-972b-b1f1c4c257bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:37.002546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 848/1681] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:36 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:37.008438 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7761f087-8373-4e58-a64e-6cafa39c56f7 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:37.042877 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 834/1682] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:44:37.066084 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0157589-481c-4c61-972b-b1f1c4c257bd None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:37.070194 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-c0157589-481c-4c61-972b-b1f1c4c257bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:37.161063 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7761f087-8373-4e58-a64e-6cafa39c56f7 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:37.174348 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:37.181819 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26994ce-afbb-49a3-8716-07440b0e4505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:37.183350 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0157589-481c-4c61-972b-b1f1c4c257bd None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:37.190128 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26994ce-afbb-49a3-8716-07440b0e4505 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:37.190128 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26994ce-afbb-49a3-8716-07440b0e4505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:37.190374 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26994ce-afbb-49a3-8716-07440b0e4505 admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:37.191071 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26994ce-afbb-49a3-8716-07440b0e4505 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:37.193304 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2885159-7a2b-4013-9e8f-3f6678d63701 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:37.194097 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26994ce-afbb-49a3-8716-07440b0e4505 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:37.194223 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2885159-7a2b-4013-9e8f-3f6678d63701 service nova] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:37.194475 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2885159-7a2b-4013-9e8f-3f6678d63701 service nova] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:37.194858 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2885159-7a2b-4013-9e8f-3f6678d63701 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:37.208038 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2885159-7a2b-4013-9e8f-3f6678d63701 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:37.211534 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2885159-7a2b-4013-9e8f-3f6678d63701 service nova] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:37.239543 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 835/1683] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:44:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:44:37.377308 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 849/1684] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:44:37 2022] POST /identity/v3/projects => generated 387 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:44:37.382157 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2885159-7a2b-4013-9e8f-3f6678d63701 service nova] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:37.409114 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2885159-7a2b-4013-9e8f-3f6678d63701 service nova] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:37.501317 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c6210c-382e-4847-8e1d-35ae9e9b0eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:37.501984 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6210c-382e-4847-8e1d-35ae9e9b0eb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:37.502280 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6210c-382e-4847-8e1d-35ae9e9b0eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:37.502623 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6210c-382e-4847-8e1d-35ae9e9b0eb1 admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:37.503351 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c6210c-382e-4847-8e1d-35ae9e9b0eb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:37.506536 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c6210c-382e-4847-8e1d-35ae9e9b0eb1 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:37.639538 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 836/1685] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:44:37 2022] POST /identity/v3/users => generated 300 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:44:37.641349 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26994ce-afbb-49a3-8716-07440b0e4505 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:37.677753 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26994ce-afbb-49a3-8716-07440b0e4505 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:37.705115 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc9be6d-3923-4900-85fb-7677eb0bfd2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:37.705902 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9be6d-3923-4900-85fb-7677eb0bfd2b admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:37.706209 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9be6d-3923-4900-85fb-7677eb0bfd2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:37.706795 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9be6d-3923-4900-85fb-7677eb0bfd2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:37.707150 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc9be6d-3923-4900-85fb-7677eb0bfd2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:37.709437 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc9be6d-3923-4900-85fb-7677eb0bfd2b admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:37.745533 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 850/1686] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:44:37 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:44:37.749731 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c6210c-382e-4847-8e1d-35ae9e9b0eb1 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:37.763459 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c6210c-382e-4847-8e1d-35ae9e9b0eb1 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:37.795219 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4600a208-3bbe-4fab-a460-1ed4e15c6a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:37.798767 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4600a208-3bbe-4fab-a460-1ed4e15c6a9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:37.799337 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4600a208-3bbe-4fab-a460-1ed4e15c6a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:37.799720 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4600a208-3bbe-4fab-a460-1ed4e15c6a9e admin admin] PATH_INFO: `/v3/projects/4c5c52c1a2324a32849cf461073d95fe/users/733c2b9f0c8f4a82b1673b7d8fb39015/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:37.857560 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4600a208-3bbe-4fab-a460-1ed4e15c6a9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c5c52c1a2324a32849cf461073d95fe, user_id=733c2b9f0c8f4a82b1673b7d8fb39015, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:37.863380 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4600a208-3bbe-4fab-a460-1ed4e15c6a9e admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:37.926928 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa6a14-7494-4534-bce2-3c41a0741be0 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:37.926928 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa6a14-7494-4534-bce2-3c41a0741be0 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:37.926928 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa6a14-7494-4534-bce2-3c41a0741be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:37.983066 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-22aa6a14-7494-4534-bce2-3c41a0741be0 None None] MFA Rules not processed for user `9bda9f6f00c74112a54573c3905ad1bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:37.987354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 837/1687] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:44:37 2022] PUT /identity/v3/projects/4c5c52c1a2324a32849cf461073d95fe/users/733c2b9f0c8f4a82b1673b7d8fb39015/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:44:37.994220 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4600a208-3bbe-4fab-a460-1ed4e15c6a9e admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:37.999393 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-22aa6a14-7494-4534-bce2-3c41a0741be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:38.137302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4600a208-3bbe-4fab-a460-1ed4e15c6a9e admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:38.145720 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1019f1-153e-4c12-9f1b-0cf911eb18c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:38.146429 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1019f1-153e-4c12-9f1b-0cf911eb18c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:38.146715 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1019f1-153e-4c12-9f1b-0cf911eb18c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:38.146989 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1019f1-153e-4c12-9f1b-0cf911eb18c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:38.148229 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1019f1-153e-4c12-9f1b-0cf911eb18c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:38.149312 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1019f1-153e-4c12-9f1b-0cf911eb18c1 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:38.184256 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 838/1688] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:44:37 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:44:38.187726 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1019f1-153e-4c12-9f1b-0cf911eb18c1 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:38.212577 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 851/1689] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:38.220850 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1019f1-153e-4c12-9f1b-0cf911eb18c1 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:38.222326 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b63f28-5dde-4dd8-ac49-b2b547f54f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:38.224176 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b63f28-5dde-4dd8-ac49-b2b547f54f19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:38.224717 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b63f28-5dde-4dd8-ac49-b2b547f54f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:38.225195 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b63f28-5dde-4dd8-ac49-b2b547f54f19 admin admin] PATH_INFO: `/v3/projects/4c5c52c1a2324a32849cf461073d95fe/users/733c2b9f0c8f4a82b1673b7d8fb39015/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:38.231363 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475dece-16dd-48a4-915c-201b05d75bdb None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:38.231689 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475dece-16dd-48a4-915c-201b05d75bdb None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:38.231921 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475dece-16dd-48a4-915c-201b05d75bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:38.233790 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b63f28-5dde-4dd8-ac49-b2b547f54f19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c5c52c1a2324a32849cf461073d95fe, user_id=733c2b9f0c8f4a82b1673b7d8fb39015, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:38.241335 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b63f28-5dde-4dd8-ac49-b2b547f54f19 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:38.253857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-2475dece-16dd-48a4-915c-201b05d75bdb None None] MFA Rules not processed for user `9bda9f6f00c74112a54573c3905ad1bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:38.255431 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-2475dece-16dd-48a4-915c-201b05d75bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:38.277425 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 852/1690] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:38.283065 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2475dece-16dd-48a4-915c-201b05d75bdb None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:38.326509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 839/1691] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:44:38 2022] PUT /identity/v3/projects/4c5c52c1a2324a32849cf461073d95fe/users/733c2b9f0c8f4a82b1673b7d8fb39015/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:44:38.327687 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001acc2b-ca6a-4db8-b529-d53cdd662881 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:38.327976 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001acc2b-ca6a-4db8-b529-d53cdd662881 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:38.328630 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001acc2b-ca6a-4db8-b529-d53cdd662881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:38.363358 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:38.381656 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-001acc2b-ca6a-4db8-b529-d53cdd662881 None None] MFA Rules not processed for user `733c2b9f0c8f4a82b1673b7d8fb39015`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:38.383050 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-001acc2b-ca6a-4db8-b529-d53cdd662881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:38.450586 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2475dece-16dd-48a4-915c-201b05d75bdb None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:38.538767 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 840/1692] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:38 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:44:38.563293 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001acc2b-ca6a-4db8-b529-d53cdd662881 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:38.647364 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3c1987-4822-476f-b903-dd8e51d8fc42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:38.655117 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c1987-4822-476f-b903-dd8e51d8fc42 service nova] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:38.656128 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c1987-4822-476f-b903-dd8e51d8fc42 service nova] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:38.656494 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c1987-4822-476f-b903-dd8e51d8fc42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:38.770758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001acc2b-ca6a-4db8-b529-d53cdd662881 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:38.803446 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45a9d76-1590-41c9-bb7c-ce0be3081cd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:38.803446 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a9d76-1590-41c9-bb7c-ce0be3081cd9 service cinder] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:38.803446 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a9d76-1590-41c9-bb7c-ce0be3081cd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:38.803446 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a9d76-1590-41c9-bb7c-ce0be3081cd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:38.812796 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45a9d76-1590-41c9-bb7c-ce0be3081cd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:38.814335 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45a9d76-1590-41c9-bb7c-ce0be3081cd9 service cinder] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:38.854650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 841/1693] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:44:38 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:44:38.859983 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3c1987-4822-476f-b903-dd8e51d8fc42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:38.875374 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3c1987-4822-476f-b903-dd8e51d8fc42 service nova] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:38.915731 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 853/1694] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:44:38 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 630 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:44:39.015125 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45a9d76-1590-41c9-bb7c-ce0be3081cd9 service cinder] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:39.133584 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:39.267572 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45a9d76-1590-41c9-bb7c-ce0be3081cd9 service cinder] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:39.296073 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f15e07e-f05c-4517-b425-ead7d5d8bc49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:39.296805 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f15e07e-f05c-4517-b425-ead7d5d8bc49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:39.303320 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f15e07e-f05c-4517-b425-ead7d5d8bc49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:39.303625 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f15e07e-f05c-4517-b425-ead7d5d8bc49 admin admin] PATH_INFO: `/v3/users/9efb2e94f7324d56b1a2cee06d2e9267` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:39.309056 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f15e07e-f05c-4517-b425-ead7d5d8bc49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9efb2e94f7324d56b1a2cee06d2e9267)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:39.319216 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f15e07e-f05c-4517-b425-ead7d5d8bc49 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:39.670767 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-8f15e07e-f05c-4517-b425-ead7d5d8bc49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9efb2e94f7324d56b1a2cee06d2e9267'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:39.703132 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-8f15e07e-f05c-4517-b425-ead7d5d8bc49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9efb2e94f7324d56b1a2cee06d2e9267'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:39.707863 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-8f15e07e-f05c-4517-b425-ead7d5d8bc49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9efb2e94f7324d56b1a2cee06d2e9267'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:39.880928 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-8f15e07e-f05c-4517-b425-ead7d5d8bc49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9efb2e94f7324d56b1a2cee06d2e9267'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:39.943182 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-8f15e07e-f05c-4517-b425-ead7d5d8bc49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9efb2e94f7324d56b1a2cee06d2e9267'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:39.979186 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 842/1695] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:44:39 2022] DELETE /identity/v3/users/9efb2e94f7324d56b1a2cee06d2e9267 => generated 0 bytes in 962 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:44:39.981290 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e612f-1cba-4eca-8461-c77f68c672ae None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:39.981794 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e612f-1cba-4eca-8461-c77f68c672ae None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:39.981971 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e612f-1cba-4eca-8461-c77f68c672ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:40.025682 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-075e612f-1cba-4eca-8461-c77f68c672ae None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:40.029418 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-075e612f-1cba-4eca-8461-c77f68c672ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:40.381325 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 854/1696] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:40.394441 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:40.408029 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b5e29-bd87-4262-a1a0-a6087d888c38 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:40.408678 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b5e29-bd87-4262-a1a0-a6087d888c38 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:40.409151 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b5e29-bd87-4262-a1a0-a6087d888c38 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:40.412142 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 843/1697] 10.222.0.15 () {54 vars in 912 bytes} [Wed Aug 24 05:44:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:44:40.423504 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e8a23-2c6d-4fd2-a9d0-da8eda228cd2 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:40.424270 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e8a23-2c6d-4fd2-a9d0-da8eda228cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:40.424697 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e8a23-2c6d-4fd2-a9d0-da8eda228cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:40.425779 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3b5e29-bd87-4262-a1a0-a6087d888c38 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:40.429663 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-7d3b5e29-bd87-4262-a1a0-a6087d888c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:40.543705 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-eb1e8a23-2c6d-4fd2-a9d0-da8eda228cd2 None None] MFA Rules not processed for user `b1ecd9b302b64ecf81d7e3d3356ddf7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:40.548220 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-eb1e8a23-2c6d-4fd2-a9d0-da8eda228cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:40.595171 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3b5e29-bd87-4262-a1a0-a6087d888c38 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:40.597614 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ec8fc-d4d9-4dfa-b282-728d21cfccc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:40.598442 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ec8fc-d4d9-4dfa-b282-728d21cfccc8 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:40.601206 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ec8fc-d4d9-4dfa-b282-728d21cfccc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:40.601655 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ec8fc-d4d9-4dfa-b282-728d21cfccc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:40.614027 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44ec8fc-d4d9-4dfa-b282-728d21cfccc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:40.617292 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44ec8fc-d4d9-4dfa-b282-728d21cfccc8 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:40.631672 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 844/1698] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:44:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:44:40.685444 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 855/1699] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Aug 24 05:44:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:40.716212 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ec8fc-d4d9-4dfa-b282-728d21cfccc8 service neutron] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:40.728721 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ec8fc-d4d9-4dfa-b282-728d21cfccc8 service neutron] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:40.761124 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9931920-888e-4fea-ad7e-b4d78e03aae0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:40.761771 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9931920-888e-4fea-ad7e-b4d78e03aae0 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:40.763171 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9931920-888e-4fea-ad7e-b4d78e03aae0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:40.763679 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9931920-888e-4fea-ad7e-b4d78e03aae0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:40.781997 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9931920-888e-4fea-ad7e-b4d78e03aae0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:40.784317 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9931920-888e-4fea-ad7e-b4d78e03aae0 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:40.812297 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 845/1700] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:44:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:44:41.167938 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:41.464556 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa8c07-f40a-470d-b6bf-eb7acb0e059a None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:41.465507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa8c07-f40a-470d-b6bf-eb7acb0e059a None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:41.477577 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa8c07-f40a-470d-b6bf-eb7acb0e059a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:41.557084 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-e5fa8c07-f40a-470d-b6bf-eb7acb0e059a None None] MFA Rules not processed for user `926115c2e54741e6a3aca02437309b8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:41.559544 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-e5fa8c07-f40a-470d-b6bf-eb7acb0e059a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:41.868896 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 856/1701] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:41 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 413 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:41.881155 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c693599-ef54-42cb-bc22-1aefe2c00c07 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:41.881713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c693599-ef54-42cb-bc22-1aefe2c00c07 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:41.882216 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c693599-ef54-42cb-bc22-1aefe2c00c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:41.915821 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-0c693599-ef54-42cb-bc22-1aefe2c00c07 None None] MFA Rules not processed for user `926115c2e54741e6a3aca02437309b8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:41.917314 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-0c693599-ef54-42cb-bc22-1aefe2c00c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:41.928685 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 846/1702] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:41 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:41.953962 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa8c07-f40a-470d-b6bf-eb7acb0e059a None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:41.956923 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-e5fa8c07-f40a-470d-b6bf-eb7acb0e059a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:42.024001 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c693599-ef54-42cb-bc22-1aefe2c00c07 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:42.026296 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-0c693599-ef54-42cb-bc22-1aefe2c00c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:42.086299 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c693599-ef54-42cb-bc22-1aefe2c00c07 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:42.103268 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29333f7-2f33-41ba-b8b3-e97a057c6c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:42.104351 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29333f7-2f33-41ba-b8b3-e97a057c6c54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:42.105040 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29333f7-2f33-41ba-b8b3-e97a057c6c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:42.105523 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29333f7-2f33-41ba-b8b3-e97a057c6c54 admin admin] PATH_INFO: `/v3/projects/937bc794db7940d0812728d3908701ed` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:42.107014 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29333f7-2f33-41ba-b8b3-e97a057c6c54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=937bc794db7940d0812728d3908701ed)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:42.110541 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29333f7-2f33-41ba-b8b3-e97a057c6c54 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:42.189000 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa8c07-f40a-470d-b6bf-eb7acb0e059a None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:42.208132 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f55c9f-abb4-4247-a1fd-c4a119711bba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:42.209070 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f55c9f-abb4-4247-a1fd-c4a119711bba service nova] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:42.209523 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f55c9f-abb4-4247-a1fd-c4a119711bba service nova] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:42.209948 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f55c9f-abb4-4247-a1fd-c4a119711bba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:42.226385 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f55c9f-abb4-4247-a1fd-c4a119711bba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:42.228636 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f55c9f-abb4-4247-a1fd-c4a119711bba service nova] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:42.247318 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 857/1703] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:44:41 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:44:42.250730 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b790dbea-e34b-4766-b4a9-11f6a9331644 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:42.251600 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b790dbea-e34b-4766-b4a9-11f6a9331644 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:42.252325 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b790dbea-e34b-4766-b4a9-11f6a9331644 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:42.255862 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 858/1704] 10.222.0.15 () {54 vars in 961 bytes} [Wed Aug 24 05:44:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:44:42.269690 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134806b8-2db6-456d-88b4-cc9c0021a79d None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:42.270076 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134806b8-2db6-456d-88b4-cc9c0021a79d None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:42.270432 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134806b8-2db6-456d-88b4-cc9c0021a79d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:42.348530 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-f29333f7-2f33-41ba-b8b3-e97a057c6c54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '937bc794db7940d0812728d3908701ed', 'request_id': 'req-f29333f7-2f33-41ba-b8b3-e97a057c6c54'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:42.349403 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-134806b8-2db6-456d-88b4-cc9c0021a79d None None] MFA Rules not processed for user `6778b7c5ad19484b877494f143050405`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:42.351366 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-134806b8-2db6-456d-88b4-cc9c0021a79d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:42.360070 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-f29333f7-2f33-41ba-b8b3-e97a057c6c54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '937bc794db7940d0812728d3908701ed', 'request_id': 'req-f29333f7-2f33-41ba-b8b3-e97a057c6c54'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:42.435651 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-f29333f7-2f33-41ba-b8b3-e97a057c6c54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '937bc794db7940d0812728d3908701ed', 'request_id': 'req-f29333f7-2f33-41ba-b8b3-e97a057c6c54'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:42.455094 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-f29333f7-2f33-41ba-b8b3-e97a057c6c54 admin admin] The token cache is being invalidate because project 937bc794db7940d0812728d3908701ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:44:42.455387 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-f29333f7-2f33-41ba-b8b3-e97a057c6c54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:42.460976 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-f29333f7-2f33-41ba-b8b3-e97a057c6c54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:42.464037 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 847/1705] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:44:42 2022] DELETE /identity/v3/projects/937bc794db7940d0812728d3908701ed => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:44:42.468418 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29333f7-2f33-41ba-b8b3-e97a057c6c54 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:42.471448 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-f29333f7-2f33-41ba-b8b3-e97a057c6c54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:42.476369 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:42.541670 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 859/1706] 10.222.0.15 () {58 vars in 1058 bytes} [Wed Aug 24 05:44:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:42.560532 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134806b8-2db6-456d-88b4-cc9c0021a79d None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:42.574423 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134806b8-2db6-456d-88b4-cc9c0021a79d None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:42.616848 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60a5a9-ed65-4a00-922f-ca77fb564b84 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6778b7c5ad19484b877494f143050405', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:42.621231 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60a5a9-ed65-4a00-922f-ca77fb564b84 service placement] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:42.621231 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60a5a9-ed65-4a00-922f-ca77fb564b84 service placement] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:42.621231 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60a5a9-ed65-4a00-922f-ca77fb564b84 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:42.642940 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-5e60a5a9-ed65-4a00-922f-ca77fb564b84 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:42.722150 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29333f7-2f33-41ba-b8b3-e97a057c6c54 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:42.740574 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c38f84-8ba1-47bf-832e-e4b115184e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:42.741667 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c38f84-8ba1-47bf-832e-e4b115184e45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:42.741958 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c38f84-8ba1-47bf-832e-e4b115184e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:42.742242 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c38f84-8ba1-47bf-832e-e4b115184e45 admin admin] PATH_INFO: `/v3/users/733c2b9f0c8f4a82b1673b7d8fb39015` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:42.746727 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c38f84-8ba1-47bf-832e-e4b115184e45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=733c2b9f0c8f4a82b1673b7d8fb39015)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:42.747007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e60a5a9-ed65-4a00-922f-ca77fb564b84 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:42.748423 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e60a5a9-ed65-4a00-922f-ca77fb564b84 service placement] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:42.752122 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c38f84-8ba1-47bf-832e-e4b115184e45 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:42.790558 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 860/1707] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:44:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:44:43.061390 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-31c38f84-8ba1-47bf-832e-e4b115184e45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '733c2b9f0c8f4a82b1673b7d8fb39015'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:43.076180 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-31c38f84-8ba1-47bf-832e-e4b115184e45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '733c2b9f0c8f4a82b1673b7d8fb39015'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:43.079311 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-31c38f84-8ba1-47bf-832e-e4b115184e45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '733c2b9f0c8f4a82b1673b7d8fb39015'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:43.172911 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-31c38f84-8ba1-47bf-832e-e4b115184e45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '733c2b9f0c8f4a82b1673b7d8fb39015'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:43.180303 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58168244-a3a5-4c28-912c-a2057fef29d3 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:43.184983 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58168244-a3a5-4c28-912c-a2057fef29d3 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:43.184983 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58168244-a3a5-4c28-912c-a2057fef29d3 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:43.184983 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 861/1708] 10.222.0.15 () {54 vars in 912 bytes} [Wed Aug 24 05:44:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:44:43.193302 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f36dde-28a9-4be3-b1ca-ee2e57e9ea71 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:43.194277 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f36dde-28a9-4be3-b1ca-ee2e57e9ea71 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:43.194711 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f36dde-28a9-4be3-b1ca-ee2e57e9ea71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:43.206889 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-31c38f84-8ba1-47bf-832e-e4b115184e45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '733c2b9f0c8f4a82b1673b7d8fb39015'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:43.224973 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 848/1709] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:44:42 2022] DELETE /identity/v3/users/733c2b9f0c8f4a82b1673b7d8fb39015 => generated 0 bytes in 757 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:44:43.240566 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-e3f36dde-28a9-4be3-b1ca-ee2e57e9ea71 None None] MFA Rules not processed for user `b1ecd9b302b64ecf81d7e3d3356ddf7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:43.241992 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-e3f36dde-28a9-4be3-b1ca-ee2e57e9ea71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:43.321714 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:43.420768 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 862/1710] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Aug 24 05:44:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:44:43.429536 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c38f84-8ba1-47bf-832e-e4b115184e45 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:43.431170 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-31c38f84-8ba1-47bf-832e-e4b115184e45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:43.453597 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c38f84-8ba1-47bf-832e-e4b115184e45 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:43.460060 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c15de-86d7-4679-9623-ff33d53f88da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:43.461157 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c15de-86d7-4679-9623-ff33d53f88da service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:43.461623 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c15de-86d7-4679-9623-ff33d53f88da service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:43.462089 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c15de-86d7-4679-9623-ff33d53f88da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:43.476270 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1c15de-86d7-4679-9623-ff33d53f88da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:43.478089 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1c15de-86d7-4679-9623-ff33d53f88da service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:43.489552 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 849/1711] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:44:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:44:44.096077 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab17b3-2dfb-4594-a45e-3ef1e1852d4e None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:44.096585 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab17b3-2dfb-4594-a45e-3ef1e1852d4e None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:44.097333 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab17b3-2dfb-4594-a45e-3ef1e1852d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:44.149881 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-0eab17b3-2dfb-4594-a45e-3ef1e1852d4e None None] MFA Rules not processed for user `d3e6e20d5cd44444b43b1e7abb240f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:44.154434 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-0eab17b3-2dfb-4594-a45e-3ef1e1852d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:44.341735 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 863/1712] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:44.508097 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c15de-86d7-4679-9623-ff33d53f88da service neutron] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:44.510019 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-6d1c15de-86d7-4679-9623-ff33d53f88da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:44.607002 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d729e-494e-41c4-a423-bb0631135b19 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:44.608054 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d729e-494e-41c4-a423-bb0631135b19 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:44.608795 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d729e-494e-41c4-a423-bb0631135b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:44.675637 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-3c1d729e-494e-41c4-a423-bb0631135b19 None None] MFA Rules not processed for user `926115c2e54741e6a3aca02437309b8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:44.682411 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-3c1d729e-494e-41c4-a423-bb0631135b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:44.705396 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:44.779341 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c15de-86d7-4679-9623-ff33d53f88da service neutron] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:44.793887 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84730a9b-cf71-4841-af91-945d4ae199d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:44.797604 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84730a9b-cf71-4841-af91-945d4ae199d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:44.798338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84730a9b-cf71-4841-af91-945d4ae199d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:44.799602 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84730a9b-cf71-4841-af91-945d4ae199d4 admin admin] PATH_INFO: `/v3/projects/4c5c52c1a2324a32849cf461073d95fe` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:44.802440 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84730a9b-cf71-4841-af91-945d4ae199d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c5c52c1a2324a32849cf461073d95fe)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:44.808475 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84730a9b-cf71-4841-af91-945d4ae199d4 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:44.820679 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 864/1713] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:44 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:44:44.856727 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:45.056389 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-84730a9b-cf71-4841-af91-945d4ae199d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c5c52c1a2324a32849cf461073d95fe', 'request_id': 'req-84730a9b-cf71-4841-af91-945d4ae199d4'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:45.086077 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-84730a9b-cf71-4841-af91-945d4ae199d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c5c52c1a2324a32849cf461073d95fe', 'request_id': 'req-84730a9b-cf71-4841-af91-945d4ae199d4'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:45.191854 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-84730a9b-cf71-4841-af91-945d4ae199d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c5c52c1a2324a32849cf461073d95fe', 'request_id': 'req-84730a9b-cf71-4841-af91-945d4ae199d4'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:45.252033 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-84730a9b-cf71-4841-af91-945d4ae199d4 admin admin] The token cache is being invalidate because project 4c5c52c1a2324a32849cf461073d95fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:44:45.255053 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-84730a9b-cf71-4841-af91-945d4ae199d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:45.256814 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-84730a9b-cf71-4841-af91-945d4ae199d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:45.260890 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 850/1714] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:44:44 2022] DELETE /identity/v3/projects/4c5c52c1a2324a32849cf461073d95fe => generated 0 bytes in 753 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:44:45.265583 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82bbe85-cde2-48e1-b7a2-a4b9dd41169e None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:45.266196 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82bbe85-cde2-48e1-b7a2-a4b9dd41169e None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:45.266696 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82bbe85-cde2-48e1-b7a2-a4b9dd41169e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:45.298723 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-d82bbe85-cde2-48e1-b7a2-a4b9dd41169e None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:45.300434 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-d82bbe85-cde2-48e1-b7a2-a4b9dd41169e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:45.549899 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 865/1715] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:45.555856 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47bb57-d220-49b8-80dd-9e1efecff9a9 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:45.556761 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47bb57-d220-49b8-80dd-9e1efecff9a9 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:45.557274 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47bb57-d220-49b8-80dd-9e1efecff9a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:45.605069 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-0d47bb57-d220-49b8-80dd-9e1efecff9a9 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:45.608883 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-0d47bb57-d220-49b8-80dd-9e1efecff9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:45.627719 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 851/1716] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:45.633272 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82bbe85-cde2-48e1-b7a2-a4b9dd41169e None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:45.647936 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82bbe85-cde2-48e1-b7a2-a4b9dd41169e None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:45.654069 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f03195-2080-4401-8acf-d54db02faaba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:45.654875 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f03195-2080-4401-8acf-d54db02faaba admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:45.660998 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f03195-2080-4401-8acf-d54db02faaba admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:45.660998 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f03195-2080-4401-8acf-d54db02faaba admin admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:45.660998 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f03195-2080-4401-8acf-d54db02faaba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:44:45.660998 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f03195-2080-4401-8acf-d54db02faaba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:45.660998 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f03195-2080-4401-8acf-d54db02faaba admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:45.681492 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 866/1717] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:44:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:44:45.682639 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d47bb57-d220-49b8-80dd-9e1efecff9a9 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:45.704676 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d47bb57-d220-49b8-80dd-9e1efecff9a9 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:45.713526 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2192e9c7-a9c2-4c7c-b474-ff112a471703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:45.715641 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2192e9c7-a9c2-4c7c-b474-ff112a471703 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:45.716211 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2192e9c7-a9c2-4c7c-b474-ff112a471703 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:45.718881 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2192e9c7-a9c2-4c7c-b474-ff112a471703 admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:45.720564 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2192e9c7-a9c2-4c7c-b474-ff112a471703 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:45.725562 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2192e9c7-a9c2-4c7c-b474-ff112a471703 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:45.925142 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 852/1718] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:44:45 2022] POST /identity/v3/projects => generated 409 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:44:45.926903 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f03195-2080-4401-8acf-d54db02faaba admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:45.950226 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f03195-2080-4401-8acf-d54db02faaba admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:45.968041 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:46.052498 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5fa31a-3666-47ca-adfe-a7873879dc19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:46.054522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5fa31a-3666-47ca-adfe-a7873879dc19 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:46.055031 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5fa31a-3666-47ca-adfe-a7873879dc19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:46.055456 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5fa31a-3666-47ca-adfe-a7873879dc19 admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:46.057608 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5fa31a-3666-47ca-adfe-a7873879dc19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:46.060854 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5fa31a-3666-47ca-adfe-a7873879dc19 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:46.270283 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 867/1719] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:44:45 2022] POST /identity/v3/users => generated 305 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:44:46.270844 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2192e9c7-a9c2-4c7c-b474-ff112a471703 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:46.287020 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2192e9c7-a9c2-4c7c-b474-ff112a471703 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:46.293039 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6312e42b-98bb-48a2-8f7c-066b6a4ab0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:46.294733 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312e42b-98bb-48a2-8f7c-066b6a4ab0a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:46.295014 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312e42b-98bb-48a2-8f7c-066b6a4ab0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:46.295465 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312e42b-98bb-48a2-8f7c-066b6a4ab0a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:46.298103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6312e42b-98bb-48a2-8f7c-066b6a4ab0a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:46.300243 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6312e42b-98bb-48a2-8f7c-066b6a4ab0a9 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:46.312664 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 853/1720] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:44:46 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:44:46.316991 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5fa31a-3666-47ca-adfe-a7873879dc19 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:46.333798 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5fa31a-3666-47ca-adfe-a7873879dc19 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:46.359249 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00ebca2-2709-4948-95be-66ccc90ffd52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:46.359249 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ebca2-2709-4948-95be-66ccc90ffd52 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:46.359249 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ebca2-2709-4948-95be-66ccc90ffd52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:46.359249 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ebca2-2709-4948-95be-66ccc90ffd52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:46.362390 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00ebca2-2709-4948-95be-66ccc90ffd52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:46.365145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00ebca2-2709-4948-95be-66ccc90ffd52 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:46.394003 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 868/1721] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:44:46 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:44:46.398466 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6312e42b-98bb-48a2-8f7c-066b6a4ab0a9 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:46.415238 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6312e42b-98bb-48a2-8f7c-066b6a4ab0a9 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:46.421434 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c993b-5de3-476f-ae9d-57ebf236e275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:46.422595 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c993b-5de3-476f-ae9d-57ebf236e275 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:46.423006 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c993b-5de3-476f-ae9d-57ebf236e275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:46.423350 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c993b-5de3-476f-ae9d-57ebf236e275 admin admin] PATH_INFO: `/v3/projects/1b6112a8e36041a8bd25229c86f3f316/users/fd0e275e4a4043af8f57e118e6326bd3/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:46.445681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5c993b-5de3-476f-ae9d-57ebf236e275 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b6112a8e36041a8bd25229c86f3f316, user_id=fd0e275e4a4043af8f57e118e6326bd3, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:46.449135 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5c993b-5de3-476f-ae9d-57ebf236e275 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:46.548636 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 854/1722] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:44:46 2022] PUT /identity/v3/projects/1b6112a8e36041a8bd25229c86f3f316/users/fd0e275e4a4043af8f57e118e6326bd3/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:44:46.570394 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba5e3e-6f8f-4cc5-b638-4969e8e4eaf7 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:46.577001 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba5e3e-6f8f-4cc5-b638-4969e8e4eaf7 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:46.577001 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba5e3e-6f8f-4cc5-b638-4969e8e4eaf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:46.709030 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-2bba5e3e-6f8f-4cc5-b638-4969e8e4eaf7 None None] MFA Rules not processed for user `fd0e275e4a4043af8f57e118e6326bd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:46.713012 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-2bba5e3e-6f8f-4cc5-b638-4969e8e4eaf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:46.891280 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 869/1723] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:46 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:46.906712 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c993b-5de3-476f-ae9d-57ebf236e275 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:47.133795 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c993b-5de3-476f-ae9d-57ebf236e275 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:47.147638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb6700a-8373-4047-a986-ae55e885ce8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:47.148945 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb6700a-8373-4047-a986-ae55e885ce8a admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:47.150093 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb6700a-8373-4047-a986-ae55e885ce8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:47.150293 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb6700a-8373-4047-a986-ae55e885ce8a admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:47.151003 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb6700a-8373-4047-a986-ae55e885ce8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:47.153723 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb6700a-8373-4047-a986-ae55e885ce8a admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:47.247570 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 855/1724] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:44:46 2022] POST /identity/v3/projects => generated 407 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:44:47.252162 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba5e3e-6f8f-4cc5-b638-4969e8e4eaf7 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:47.277885 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba5e3e-6f8f-4cc5-b638-4969e8e4eaf7 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:47.379260 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4acc3e-b3aa-4218-a01b-a978b320d917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:47.380006 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4acc3e-b3aa-4218-a01b-a978b320d917 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:47.381523 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4acc3e-b3aa-4218-a01b-a978b320d917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:47.385701 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4acc3e-b3aa-4218-a01b-a978b320d917 admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:47.385701 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4acc3e-b3aa-4218-a01b-a978b320d917 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:47.385701 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4acc3e-b3aa-4218-a01b-a978b320d917 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:47.499073 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 870/1725] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:44:47 2022] POST /identity/v3/users => generated 310 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:44:47.504174 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb6700a-8373-4047-a986-ae55e885ce8a admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:47.529035 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb6700a-8373-4047-a986-ae55e885ce8a admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:47.534272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e667c4c5-3dbe-403e-8272-a0ab6289cd23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:47.535968 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e667c4c5-3dbe-403e-8272-a0ab6289cd23 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:47.536699 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e667c4c5-3dbe-403e-8272-a0ab6289cd23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:47.537158 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e667c4c5-3dbe-403e-8272-a0ab6289cd23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:47.538014 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e667c4c5-3dbe-403e-8272-a0ab6289cd23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:47.540583 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e667c4c5-3dbe-403e-8272-a0ab6289cd23 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:47.554722 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 856/1726] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:44:47 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:44:47.559721 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4acc3e-b3aa-4218-a01b-a978b320d917 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:47.573902 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4acc3e-b3aa-4218-a01b-a978b320d917 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:47.581849 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd2a967-b13e-4f3a-aa20-7da4ea516013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:47.583304 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2a967-b13e-4f3a-aa20-7da4ea516013 admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:47.583825 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2a967-b13e-4f3a-aa20-7da4ea516013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:47.584112 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2a967-b13e-4f3a-aa20-7da4ea516013 admin admin] PATH_INFO: `/v3/projects/778c1fe92d774ea88732bc38f33d7b22/users/4555f303a71a4566bdea2ceb382abba5/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:47.607745 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:47.612809 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd2a967-b13e-4f3a-aa20-7da4ea516013 admin admin] RBAC: Authorizing `identity:create_grant(project_id=778c1fe92d774ea88732bc38f33d7b22, user_id=4555f303a71a4566bdea2ceb382abba5, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:47.618104 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd2a967-b13e-4f3a-aa20-7da4ea516013 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:47.732680 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:47.757630 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 871/1727] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:44:47 2022] PUT /identity/v3/projects/778c1fe92d774ea88732bc38f33d7b22/users/4555f303a71a4566bdea2ceb382abba5/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:44:47.761638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e667c4c5-3dbe-403e-8272-a0ab6289cd23 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:47.790768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e667c4c5-3dbe-403e-8272-a0ab6289cd23 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:47.990508 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe3d6c-6c3a-4f19-a8a8-4373bc4d9809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:47.990508 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe3d6c-6c3a-4f19-a8a8-4373bc4d9809 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:47.990508 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe3d6c-6c3a-4f19-a8a8-4373bc4d9809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:47.990508 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe3d6c-6c3a-4f19-a8a8-4373bc4d9809 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:47.991248 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fe3d6c-6c3a-4f19-a8a8-4373bc4d9809 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:47.992542 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fe3d6c-6c3a-4f19-a8a8-4373bc4d9809 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:48.015399 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 857/1728] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:44:47 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:44:48.023082 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd2a967-b13e-4f3a-aa20-7da4ea516013 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:48.039067 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd2a967-b13e-4f3a-aa20-7da4ea516013 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:48.045791 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c9807-7606-4130-a5de-eca9eeec5d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:48.048586 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c9807-7606-4130-a5de-eca9eeec5d7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:48.049132 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c9807-7606-4130-a5de-eca9eeec5d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:48.049619 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c9807-7606-4130-a5de-eca9eeec5d7c admin admin] PATH_INFO: `/v3/projects/778c1fe92d774ea88732bc38f33d7b22/users/4555f303a71a4566bdea2ceb382abba5/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:48.052365 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95c9807-7606-4130-a5de-eca9eeec5d7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=778c1fe92d774ea88732bc38f33d7b22, user_id=4555f303a71a4566bdea2ceb382abba5, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:48.071134 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95c9807-7606-4130-a5de-eca9eeec5d7c admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:48.091217 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 872/1729] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:44:48 2022] PUT /identity/v3/projects/778c1fe92d774ea88732bc38f33d7b22/users/4555f303a71a4566bdea2ceb382abba5/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:44:48.108770 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fdb217-3507-429b-86e7-c7f9a0cdf1a7 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:48.109654 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fdb217-3507-429b-86e7-c7f9a0cdf1a7 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:48.110362 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fdb217-3507-429b-86e7-c7f9a0cdf1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:48.231056 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-b9fdb217-3507-429b-86e7-c7f9a0cdf1a7 None None] MFA Rules not processed for user `4555f303a71a4566bdea2ceb382abba5`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:48.236342 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-b9fdb217-3507-429b-86e7-c7f9a0cdf1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:48.367030 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 858/1730] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:48 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:48.385552 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c9807-7606-4130-a5de-eca9eeec5d7c admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:48.387547 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-e95c9807-7606-4130-a5de-eca9eeec5d7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:48.474433 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c9807-7606-4130-a5de-eca9eeec5d7c admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:48.480318 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f30e586-16ca-4ee2-b115-7ab0d7a7872c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:48.482951 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f30e586-16ca-4ee2-b115-7ab0d7a7872c service cinder] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:48.482951 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f30e586-16ca-4ee2-b115-7ab0d7a7872c service cinder] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:48.482951 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f30e586-16ca-4ee2-b115-7ab0d7a7872c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:48.502228 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f30e586-16ca-4ee2-b115-7ab0d7a7872c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:48.503202 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f30e586-16ca-4ee2-b115-7ab0d7a7872c service cinder] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:48.514526 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 873/1731] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:44:48 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:44:49.104391 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fdb217-3507-429b-86e7-c7f9a0cdf1a7 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:49.114772 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fdb217-3507-429b-86e7-c7f9a0cdf1a7 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:49.118459 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ff2bc-6947-4d49-a984-8f58cc645ee9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:49.119065 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ff2bc-6947-4d49-a984-8f58cc645ee9 service cinder] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:49.119345 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ff2bc-6947-4d49-a984-8f58cc645ee9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:49.119792 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ff2bc-6947-4d49-a984-8f58cc645ee9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:49.193060 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4ff2bc-6947-4d49-a984-8f58cc645ee9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:49.194849 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4ff2bc-6947-4d49-a984-8f58cc645ee9 service cinder] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:49.206098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 859/1732] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:44:49 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:44:49.253524 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:52.840250 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a70679-c96a-4c33-9123-3d3f8b7e0e5a None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:52.841139 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a70679-c96a-4c33-9123-3d3f8b7e0e5a None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:52.841566 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a70679-c96a-4c33-9123-3d3f8b7e0e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:53.027868 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-69a70679-c96a-4c33-9123-3d3f8b7e0e5a None None] MFA Rules not processed for user `604380c191c8461bbb937b8d951238bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:53.029196 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-69a70679-c96a-4c33-9123-3d3f8b7e0e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:53.205731 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 874/1733] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Aug 24 05:44:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:44:53.766611 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ff2bc-6947-4d49-a984-8f58cc645ee9 service cinder] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:53.766611 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-9c4ff2bc-6947-4d49-a984-8f58cc645ee9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:53.821238 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a70679-c96a-4c33-9123-3d3f8b7e0e5a None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:53.823342 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-69a70679-c96a-4c33-9123-3d3f8b7e0e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:53.992038 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ff2bc-6947-4d49-a984-8f58cc645ee9 service cinder] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:53.997946 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9bf4b7-d1ce-42e4-8ca7-8439fe8d8f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:54.002664 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9bf4b7-d1ce-42e4-8ca7-8439fe8d8f33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:54.002664 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9bf4b7-d1ce-42e4-8ca7-8439fe8d8f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:54.002664 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9bf4b7-d1ce-42e4-8ca7-8439fe8d8f33 admin admin] PATH_INFO: `/v3/users/4638fb875c0e4afbad0695c561db3f53` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:54.002664 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9bf4b7-d1ce-42e4-8ca7-8439fe8d8f33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4638fb875c0e4afbad0695c561db3f53)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:54.003546 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9bf4b7-d1ce-42e4-8ca7-8439fe8d8f33 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:54.013328 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:54.072748 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:54.143633 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a70679-c96a-4c33-9123-3d3f8b7e0e5a None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:54.151236 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c2bb45-e37f-4dff-b501-acb1a94e03ae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb2b6cae4a004539a95fc7dd558db255', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:54.152169 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2bb45-e37f-4dff-b501-acb1a94e03ae service swift] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:54.152875 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2bb45-e37f-4dff-b501-acb1a94e03ae service swift] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:54.168854 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2bb45-e37f-4dff-b501-acb1a94e03ae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:54.199442 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c2bb45-e37f-4dff-b501-acb1a94e03ae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:54.201162 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c2bb45-e37f-4dff-b501-acb1a94e03ae service swift] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:54.215478 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 875/1734] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Aug 24 05:44:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 05:44:54.424973 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-4d9bf4b7-d1ce-42e4-8ca7-8439fe8d8f33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4638fb875c0e4afbad0695c561db3f53'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:54.437921 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-4d9bf4b7-d1ce-42e4-8ca7-8439fe8d8f33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4638fb875c0e4afbad0695c561db3f53'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:54.444335 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-4d9bf4b7-d1ce-42e4-8ca7-8439fe8d8f33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4638fb875c0e4afbad0695c561db3f53'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:54.679318 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-4d9bf4b7-d1ce-42e4-8ca7-8439fe8d8f33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4638fb875c0e4afbad0695c561db3f53'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:54.749606 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-4d9bf4b7-d1ce-42e4-8ca7-8439fe8d8f33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4638fb875c0e4afbad0695c561db3f53'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:54.767827 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de50c1-5c5c-4593-8a6f-3bbe28c1061f None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:54.768514 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de50c1-5c5c-4593-8a6f-3bbe28c1061f None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:54.769536 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de50c1-5c5c-4593-8a6f-3bbe28c1061f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:54.770502 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 860/1735] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:44:53 2022] DELETE /identity/v3/users/4638fb875c0e4afbad0695c561db3f53 => generated 0 bytes in 1004 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:44:54.815331 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-72de50c1-5c5c-4593-8a6f-3bbe28c1061f None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:54.816793 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-72de50c1-5c5c-4593-8a6f-3bbe28c1061f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:54.998692 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 876/1736] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:44:55.051655 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9bf4b7-d1ce-42e4-8ca7-8439fe8d8f33 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:55.092362 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-4d9bf4b7-d1ce-42e4-8ca7-8439fe8d8f33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:55.322535 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9bf4b7-d1ce-42e4-8ca7-8439fe8d8f33 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:55.363694 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b36844-5a78-4b94-819a-9577f7cd5a4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:55.365137 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b36844-5a78-4b94-819a-9577f7cd5a4e service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:55.365642 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b36844-5a78-4b94-819a-9577f7cd5a4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:55.365891 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b36844-5a78-4b94-819a-9577f7cd5a4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:55.386988 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826204cd-bf44-4d36-b19a-d6af8e886df8 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:55.386988 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826204cd-bf44-4d36-b19a-d6af8e886df8 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:55.386988 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826204cd-bf44-4d36-b19a-d6af8e886df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:55.392361 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b36844-5a78-4b94-819a-9577f7cd5a4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:55.394791 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b36844-5a78-4b94-819a-9577f7cd5a4e service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:55.443283 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 861/1737] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:44:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:44:55.476948 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-826204cd-bf44-4d36-b19a-d6af8e886df8 None None] MFA Rules not processed for user `b1ecd9b302b64ecf81d7e3d3356ddf7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:55.480690 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-826204cd-bf44-4d36-b19a-d6af8e886df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:55.517122 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 877/1738] 10.222.0.40 () {58 vars in 1015 bytes} [Wed Aug 24 05:44:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:55.584876 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b36844-5a78-4b94-819a-9577f7cd5a4e service neutron] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:55.584876 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-b1b36844-5a78-4b94-819a-9577f7cd5a4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:55.593260 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826204cd-bf44-4d36-b19a-d6af8e886df8 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:55.599132 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-826204cd-bf44-4d36-b19a-d6af8e886df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:55.650468 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826204cd-bf44-4d36-b19a-d6af8e886df8 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:55.661688 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b36844-5a78-4b94-819a-9577f7cd5a4e service neutron] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:55.669044 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3931474-43e6-4a2f-aeee-9798c6dd3b28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:55.671397 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3931474-43e6-4a2f-aeee-9798c6dd3b28 service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:55.673131 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3931474-43e6-4a2f-aeee-9798c6dd3b28 service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:55.673667 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3931474-43e6-4a2f-aeee-9798c6dd3b28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:55.679322 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03fed35-0ff9-44f2-a261-113e03a9a858 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:55.682315 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fed35-0ff9-44f2-a261-113e03a9a858 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:55.682315 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fed35-0ff9-44f2-a261-113e03a9a858 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:55.691990 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fed35-0ff9-44f2-a261-113e03a9a858 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:55.720628 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3931474-43e6-4a2f-aeee-9798c6dd3b28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:55.726104 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3931474-43e6-4a2f-aeee-9798c6dd3b28 service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:55.728538 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03fed35-0ff9-44f2-a261-113e03a9a858 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:55.732705 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03fed35-0ff9-44f2-a261-113e03a9a858 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:55.769985 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 878/1739] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:44:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:44:55.778425 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 862/1740] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:44:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:44:57.145505 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3931474-43e6-4a2f-aeee-9798c6dd3b28 service neutron] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:57.147935 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-d3931474-43e6-4a2f-aeee-9798c6dd3b28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:57.188885 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3931474-43e6-4a2f-aeee-9798c6dd3b28 service neutron] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:57.202734 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0bbb55-d030-43d6-81cb-3a977d56c6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:57.204106 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0bbb55-d030-43d6-81cb-3a977d56c6f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:57.204833 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0bbb55-d030-43d6-81cb-3a977d56c6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:57.205529 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0bbb55-d030-43d6-81cb-3a977d56c6f1 admin admin] PATH_INFO: `/v3/projects/1f6545fe1acc4a73a766dd17cd26678a` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:57.209506 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0bbb55-d030-43d6-81cb-3a977d56c6f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f6545fe1acc4a73a766dd17cd26678a)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:57.214965 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0bbb55-d030-43d6-81cb-3a977d56c6f1 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:57.413123 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-ad0bbb55-d030-43d6-81cb-3a977d56c6f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f6545fe1acc4a73a766dd17cd26678a', 'request_id': 'req-ad0bbb55-d030-43d6-81cb-3a977d56c6f1'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:57.433775 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-ad0bbb55-d030-43d6-81cb-3a977d56c6f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f6545fe1acc4a73a766dd17cd26678a', 'request_id': 'req-ad0bbb55-d030-43d6-81cb-3a977d56c6f1'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:57.550115 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-ad0bbb55-d030-43d6-81cb-3a977d56c6f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f6545fe1acc4a73a766dd17cd26678a', 'request_id': 'req-ad0bbb55-d030-43d6-81cb-3a977d56c6f1'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:57.594347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-ad0bbb55-d030-43d6-81cb-3a977d56c6f1 admin admin] The token cache is being invalidate because project 1f6545fe1acc4a73a766dd17cd26678a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:44:57.594850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-ad0bbb55-d030-43d6-81cb-3a977d56c6f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:57.605206 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-ad0bbb55-d030-43d6-81cb-3a977d56c6f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:44:57.607948 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 879/1741] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:44:57 2022] DELETE /identity/v3/projects/1f6545fe1acc4a73a766dd17cd26678a => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:44:57.617171 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d109110-8b69-46a9-9fb1-49b27b05069b None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:57.621185 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d109110-8b69-46a9-9fb1-49b27b05069b None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:57.638194 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d109110-8b69-46a9-9fb1-49b27b05069b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:57.698490 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-8d109110-8b69-46a9-9fb1-49b27b05069b None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:57.700037 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-8d109110-8b69-46a9-9fb1-49b27b05069b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:57.877857 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 863/1742] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:57.891568 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa6c8b-2ea1-4210-befb-ab93da7b8725 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:57.892516 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa6c8b-2ea1-4210-befb-ab93da7b8725 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:57.893251 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa6c8b-2ea1-4210-befb-ab93da7b8725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:57.934291 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-60aa6c8b-2ea1-4210-befb-ab93da7b8725 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:57.936671 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-60aa6c8b-2ea1-4210-befb-ab93da7b8725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:57.958448 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 880/1743] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:57.966811 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d109110-8b69-46a9-9fb1-49b27b05069b None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:57.994475 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d109110-8b69-46a9-9fb1-49b27b05069b None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:58.000673 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2cec2d-0b53-43d3-a2d4-2364f0250d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:58.001450 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2cec2d-0b53-43d3-a2d4-2364f0250d15 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:58.001881 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2cec2d-0b53-43d3-a2d4-2364f0250d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:58.002317 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2cec2d-0b53-43d3-a2d4-2364f0250d15 admin admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:58.003161 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2cec2d-0b53-43d3-a2d4-2364f0250d15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:44:58.003688 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2cec2d-0b53-43d3-a2d4-2364f0250d15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:58.007298 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2cec2d-0b53-43d3-a2d4-2364f0250d15 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:58.027318 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:58.035831 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 864/1744] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:44:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:44:58.043623 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aa6c8b-2ea1-4210-befb-ab93da7b8725 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:58.063780 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aa6c8b-2ea1-4210-befb-ab93da7b8725 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:58.075817 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a656d-6c69-4dbb-8721-97593d19e765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:58.077435 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a656d-6c69-4dbb-8721-97593d19e765 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:58.078154 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a656d-6c69-4dbb-8721-97593d19e765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:58.078753 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a656d-6c69-4dbb-8721-97593d19e765 admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:58.080554 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95a656d-6c69-4dbb-8721-97593d19e765 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:58.086287 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95a656d-6c69-4dbb-8721-97593d19e765 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:58.118003 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b440f8-3819-4458-8613-ecab43090372 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:58.124266 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b440f8-3819-4458-8613-ecab43090372 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:58.125458 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b440f8-3819-4458-8613-ecab43090372 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:58.131640 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 865/1745] 10.222.0.15 () {54 vars in 921 bytes} [Wed Aug 24 05:44:58 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:44:58.153492 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16656722-d678-41ad-a22e-f8e994042130 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:58.154207 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16656722-d678-41ad-a22e-f8e994042130 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:58.154687 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16656722-d678-41ad-a22e-f8e994042130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:58.160254 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:58.197768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-16656722-d678-41ad-a22e-f8e994042130 None None] MFA Rules not processed for user `fbecffb800584c5786813accd061421b`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:58.201197 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-16656722-d678-41ad-a22e-f8e994042130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:58.299733 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 866/1746] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Aug 24 05:44:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:58.313119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16656722-d678-41ad-a22e-f8e994042130 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:58.314592 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-16656722-d678-41ad-a22e-f8e994042130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:58.329863 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16656722-d678-41ad-a22e-f8e994042130 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:58.336983 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a84b0-9c5e-49ef-a630-1e3a0bb01b13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:58.337685 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a84b0-9c5e-49ef-a630-1e3a0bb01b13 service nova] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:58.337971 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a84b0-9c5e-49ef-a630-1e3a0bb01b13 service nova] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:58.338251 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a84b0-9c5e-49ef-a630-1e3a0bb01b13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:58.348350 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48a84b0-9c5e-49ef-a630-1e3a0bb01b13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:58.350061 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48a84b0-9c5e-49ef-a630-1e3a0bb01b13 service nova] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:58.359895 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 881/1747] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:44:58 2022] POST /identity/v3/projects => generated 363 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:44:58.362252 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 867/1748] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:44:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:44:58.366978 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a656d-6c69-4dbb-8721-97593d19e765 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:58.477175 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a656d-6c69-4dbb-8721-97593d19e765 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:58.484853 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8180c-b2f8-49d8-9844-8d4bbb26b63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:58.484853 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8180c-b2f8-49d8-9844-8d4bbb26b63b admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:58.484853 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8180c-b2f8-49d8-9844-8d4bbb26b63b admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:58.484853 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8180c-b2f8-49d8-9844-8d4bbb26b63b admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:58.486101 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8180c-b2f8-49d8-9844-8d4bbb26b63b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:58.486447 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8180c-b2f8-49d8-9844-8d4bbb26b63b admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:58.641720 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 882/1749] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:44:58 2022] POST /identity/v3/users => generated 282 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:44:58.647972 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a84b0-9c5e-49ef-a630-1e3a0bb01b13 service nova] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:58.668448 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a84b0-9c5e-49ef-a630-1e3a0bb01b13 service nova] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:58.670156 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c94a08c-2b8a-4d54-8b2c-108510d8d66c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:58.670845 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c94a08c-2b8a-4d54-8b2c-108510d8d66c admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:58.671179 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c94a08c-2b8a-4d54-8b2c-108510d8d66c admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:58.671519 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c94a08c-2b8a-4d54-8b2c-108510d8d66c admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:58.672358 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c94a08c-2b8a-4d54-8b2c-108510d8d66c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:58.674298 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c94a08c-2b8a-4d54-8b2c-108510d8d66c admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:58.687744 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 868/1750] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:44:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:44:58.693191 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8180c-b2f8-49d8-9844-8d4bbb26b63b admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:58.709481 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8180c-b2f8-49d8-9844-8d4bbb26b63b admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:58.712210 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6188d5-05f6-42cf-9622-eca186594212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:58.712989 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6188d5-05f6-42cf-9622-eca186594212 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:58.713270 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6188d5-05f6-42cf-9622-eca186594212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:58.713589 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6188d5-05f6-42cf-9622-eca186594212 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:58.714372 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6188d5-05f6-42cf-9622-eca186594212 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:58.716244 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6188d5-05f6-42cf-9622-eca186594212 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:58.735529 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 883/1751] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:44:58 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:44:58.742704 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c94a08c-2b8a-4d54-8b2c-108510d8d66c admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:58.762437 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c94a08c-2b8a-4d54-8b2c-108510d8d66c admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:58.771714 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a310f20d-0ea5-4ed0-bdd7-8782a5a71afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:58.772442 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310f20d-0ea5-4ed0-bdd7-8782a5a71afd admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:58.772737 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310f20d-0ea5-4ed0-bdd7-8782a5a71afd admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:58.773022 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310f20d-0ea5-4ed0-bdd7-8782a5a71afd admin admin] PATH_INFO: `/v3/projects/40683d7c990d4b4b8c04bfe33a962975/users/860e9df346cb46f29c41d5a6f392e80b/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:58.791834 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a310f20d-0ea5-4ed0-bdd7-8782a5a71afd admin admin] RBAC: Authorizing `identity:create_grant(project_id=40683d7c990d4b4b8c04bfe33a962975, user_id=860e9df346cb46f29c41d5a6f392e80b, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:58.794679 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a310f20d-0ea5-4ed0-bdd7-8782a5a71afd admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:58.902693 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 869/1752] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:44:58 2022] PUT /identity/v3/projects/40683d7c990d4b4b8c04bfe33a962975/users/860e9df346cb46f29c41d5a6f392e80b/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:44:58.922665 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d12b7b-cc9b-45df-aa10-beb3abd490bb None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:58.922989 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d12b7b-cc9b-45df-aa10-beb3abd490bb None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:58.923493 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d12b7b-cc9b-45df-aa10-beb3abd490bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:58.957838 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-a4d12b7b-cc9b-45df-aa10-beb3abd490bb None None] MFA Rules not processed for user `860e9df346cb46f29c41d5a6f392e80b`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:58.959096 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-a4d12b7b-cc9b-45df-aa10-beb3abd490bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:59.055770 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 884/1753] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:44:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:59.061259 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a310f20d-0ea5-4ed0-bdd7-8782a5a71afd admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:59.174379 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a310f20d-0ea5-4ed0-bdd7-8782a5a71afd admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:59.181494 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f2a2a-867f-426a-af7f-559da1fcfc88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:59.183076 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f2a2a-867f-426a-af7f-559da1fcfc88 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:59.183076 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f2a2a-867f-426a-af7f-559da1fcfc88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:59.183393 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f2a2a-867f-426a-af7f-559da1fcfc88 admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:59.184307 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1f2a2a-867f-426a-af7f-559da1fcfc88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:59.187371 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1f2a2a-867f-426a-af7f-559da1fcfc88 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:59.344576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d12b7b-cc9b-45df-aa10-beb3abd490bb None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:59.350120 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 870/1754] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:44:59 2022] POST /identity/v3/projects => generated 363 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:44:59.380806 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d12b7b-cc9b-45df-aa10-beb3abd490bb None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:59.537647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae97418e-90c8-4ec6-824d-eb35fa102c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:59.540846 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97418e-90c8-4ec6-824d-eb35fa102c2f admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:59.541261 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97418e-90c8-4ec6-824d-eb35fa102c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:59.541719 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97418e-90c8-4ec6-824d-eb35fa102c2f admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:59.542924 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae97418e-90c8-4ec6-824d-eb35fa102c2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:59.546213 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae97418e-90c8-4ec6-824d-eb35fa102c2f admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:59.846007 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f2a2a-867f-426a-af7f-559da1fcfc88 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:59.848854 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 885/1755] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:44:59 2022] POST /identity/v3/users => generated 288 bytes in 505 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:44:59.887494 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f2a2a-867f-426a-af7f-559da1fcfc88 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:59.926272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f408e60-c194-478e-8f71-6518fa70a16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:59.926959 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f408e60-c194-478e-8f71-6518fa70a16d admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:59.927197 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f408e60-c194-478e-8f71-6518fa70a16d admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:59.937004 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f408e60-c194-478e-8f71-6518fa70a16d admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:59.938337 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f408e60-c194-478e-8f71-6518fa70a16d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:59.950588 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f408e60-c194-478e-8f71-6518fa70a16d admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:59.967162 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 871/1756] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:44:59 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:44:59.973743 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae97418e-90c8-4ec6-824d-eb35fa102c2f admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:59.986905 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae97418e-90c8-4ec6-824d-eb35fa102c2f admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:59.993365 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51c324e-85ce-4d32-a915-1628d1328cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:59.995588 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c324e-85ce-4d32-a915-1628d1328cac admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:59.995940 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c324e-85ce-4d32-a915-1628d1328cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:59.996226 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c324e-85ce-4d32-a915-1628d1328cac admin admin] PATH_INFO: `/v3/projects/4d0da914d5a14f10b6b1a28256c5805b/users/6d6b143b2c1640439ba4bc43e56a6118/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:00.036120 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51c324e-85ce-4d32-a915-1628d1328cac admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d0da914d5a14f10b6b1a28256c5805b, user_id=6d6b143b2c1640439ba4bc43e56a6118, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:00.041240 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51c324e-85ce-4d32-a915-1628d1328cac admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:00.150760 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 886/1757] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:44:59 2022] PUT /identity/v3/projects/4d0da914d5a14f10b6b1a28256c5805b/users/6d6b143b2c1640439ba4bc43e56a6118/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:00.175390 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f408e60-c194-478e-8f71-6518fa70a16d admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:00.195010 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f408e60-c194-478e-8f71-6518fa70a16d admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:00.304954 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc9137c-52f2-43b0-8d0b-d718a6f34d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:00.307055 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc9137c-52f2-43b0-8d0b-d718a6f34d20 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:00.307828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc9137c-52f2-43b0-8d0b-d718a6f34d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:00.320859 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc9137c-52f2-43b0-8d0b-d718a6f34d20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:00.320859 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc9137c-52f2-43b0-8d0b-d718a6f34d20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:00.320859 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc9137c-52f2-43b0-8d0b-d718a6f34d20 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:00.343959 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 872/1758] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:00 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:00.349318 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51c324e-85ce-4d32-a915-1628d1328cac admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:00.363541 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51c324e-85ce-4d32-a915-1628d1328cac admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:00.378961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f6ae79-ef2f-4b38-914e-b117d7f92799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:00.380907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f6ae79-ef2f-4b38-914e-b117d7f92799 admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:00.381409 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f6ae79-ef2f-4b38-914e-b117d7f92799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:00.381903 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f6ae79-ef2f-4b38-914e-b117d7f92799 admin admin] PATH_INFO: `/v3/projects/4d0da914d5a14f10b6b1a28256c5805b/users/6d6b143b2c1640439ba4bc43e56a6118/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:00.385813 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f6ae79-ef2f-4b38-914e-b117d7f92799 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d0da914d5a14f10b6b1a28256c5805b, user_id=6d6b143b2c1640439ba4bc43e56a6118, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:00.391515 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f6ae79-ef2f-4b38-914e-b117d7f92799 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:00.412190 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 887/1759] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:00 2022] PUT /identity/v3/projects/4d0da914d5a14f10b6b1a28256c5805b/users/6d6b143b2c1640439ba4bc43e56a6118/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:45:00.431399 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e774d-1f01-4ea9-8101-b3fee9a494ff None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:00.431890 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e774d-1f01-4ea9-8101-b3fee9a494ff None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:00.432176 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e774d-1f01-4ea9-8101-b3fee9a494ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:00.481342 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-4a2e774d-1f01-4ea9-8101-b3fee9a494ff None None] MFA Rules not processed for user `6d6b143b2c1640439ba4bc43e56a6118`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:00.483370 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-4a2e774d-1f01-4ea9-8101-b3fee9a494ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:00.597744 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:00.641462 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 873/1760] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:00 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:00.655492 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f6ae79-ef2f-4b38-914e-b117d7f92799 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:00.658110 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-86f6ae79-ef2f-4b38-914e-b117d7f92799 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:00.724455 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:00.781519 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f6ae79-ef2f-4b38-914e-b117d7f92799 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:00.787986 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3ede3f-8d0e-49c0-8b11-ddb925226b03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:00.788670 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ede3f-8d0e-49c0-8b11-ddb925226b03 service cinder] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:00.788955 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ede3f-8d0e-49c0-8b11-ddb925226b03 service cinder] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:00.789376 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ede3f-8d0e-49c0-8b11-ddb925226b03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:00.813464 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3ede3f-8d0e-49c0-8b11-ddb925226b03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:00.814725 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3ede3f-8d0e-49c0-8b11-ddb925226b03 service cinder] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:00.827846 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 888/1761] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:00 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:45:02.227268 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e774d-1f01-4ea9-8101-b3fee9a494ff None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:02.311672 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e774d-1f01-4ea9-8101-b3fee9a494ff None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:02.316927 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4bc457-7ec5-46a1-b047-7d44ac085751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:02.317648 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4bc457-7ec5-46a1-b047-7d44ac085751 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:02.317932 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4bc457-7ec5-46a1-b047-7d44ac085751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:02.318852 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4bc457-7ec5-46a1-b047-7d44ac085751 admin admin] PATH_INFO: `/v3/users/860e9df346cb46f29c41d5a6f392e80b` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:02.319741 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4bc457-7ec5-46a1-b047-7d44ac085751 admin admin] RBAC: Authorizing `identity:delete_user(user_id=860e9df346cb46f29c41d5a6f392e80b)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:02.321832 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4bc457-7ec5-46a1-b047-7d44ac085751 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:02.588086 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:02.600314 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-9f4bc457-7ec5-46a1-b047-7d44ac085751 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '860e9df346cb46f29c41d5a6f392e80b'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:02.634358 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-9f4bc457-7ec5-46a1-b047-7d44ac085751 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '860e9df346cb46f29c41d5a6f392e80b'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:02.635556 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-9f4bc457-7ec5-46a1-b047-7d44ac085751 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '860e9df346cb46f29c41d5a6f392e80b'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:03.092044 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-9f4bc457-7ec5-46a1-b047-7d44ac085751 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '860e9df346cb46f29c41d5a6f392e80b'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:03.105592 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-9f4bc457-7ec5-46a1-b047-7d44ac085751 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '860e9df346cb46f29c41d5a6f392e80b'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:03.114400 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 874/1762] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:45:02 2022] DELETE /identity/v3/users/860e9df346cb46f29c41d5a6f392e80b => generated 0 bytes in 890 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:03.119031 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbca872-44df-4090-ae33-18ee1cd991be None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:03.119815 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbca872-44df-4090-ae33-18ee1cd991be None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:03.120249 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbca872-44df-4090-ae33-18ee1cd991be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:03.151672 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-6dbca872-44df-4090-ae33-18ee1cd991be None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:03.153827 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-6dbca872-44df-4090-ae33-18ee1cd991be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:03.271816 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 889/1763] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:03.282420 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4bc457-7ec5-46a1-b047-7d44ac085751 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:03.284384 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-9f4bc457-7ec5-46a1-b047-7d44ac085751 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:03.355890 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4bc457-7ec5-46a1-b047-7d44ac085751 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:03.361231 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc045bc-da0e-46f6-8298-da87f4ce83b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:03.362277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc045bc-da0e-46f6-8298-da87f4ce83b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:03.363211 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc045bc-da0e-46f6-8298-da87f4ce83b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:03.363750 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc045bc-da0e-46f6-8298-da87f4ce83b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:03.379787 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc045bc-da0e-46f6-8298-da87f4ce83b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:03.382534 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc045bc-da0e-46f6-8298-da87f4ce83b8 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:03.408907 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 875/1764] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:45:04.557795 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbca872-44df-4090-ae33-18ee1cd991be None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:04.559584 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-6dbca872-44df-4090-ae33-18ee1cd991be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:04.599558 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbca872-44df-4090-ae33-18ee1cd991be None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:04.616237 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd53f96-58f8-4f54-8d13-2c00c877b622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:04.617207 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd53f96-58f8-4f54-8d13-2c00c877b622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:04.617429 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd53f96-58f8-4f54-8d13-2c00c877b622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:04.617921 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd53f96-58f8-4f54-8d13-2c00c877b622 admin admin] PATH_INFO: `/v3/projects/40683d7c990d4b4b8c04bfe33a962975` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:04.620150 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd53f96-58f8-4f54-8d13-2c00c877b622 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40683d7c990d4b4b8c04bfe33a962975)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:04.622770 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd53f96-58f8-4f54-8d13-2c00c877b622 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:04.749133 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-9fd53f96-58f8-4f54-8d13-2c00c877b622 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40683d7c990d4b4b8c04bfe33a962975', 'request_id': 'req-9fd53f96-58f8-4f54-8d13-2c00c877b622'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:04.759580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-9fd53f96-58f8-4f54-8d13-2c00c877b622 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40683d7c990d4b4b8c04bfe33a962975', 'request_id': 'req-9fd53f96-58f8-4f54-8d13-2c00c877b622'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:04.762045 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:INSERT=3,keystone:SELECT=93,keystone:DELETE=4 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:04.866788 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-9fd53f96-58f8-4f54-8d13-2c00c877b622 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40683d7c990d4b4b8c04bfe33a962975', 'request_id': 'req-9fd53f96-58f8-4f54-8d13-2c00c877b622'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:04.896962 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-9fd53f96-58f8-4f54-8d13-2c00c877b622 admin admin] The token cache is being invalidate because project 40683d7c990d4b4b8c04bfe33a962975 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:45:04.901263 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-9fd53f96-58f8-4f54-8d13-2c00c877b622 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:04.901263 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-9fd53f96-58f8-4f54-8d13-2c00c877b622 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:04.903089 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 890/1765] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:45:04 2022] DELETE /identity/v3/projects/40683d7c990d4b4b8c04bfe33a962975 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:04.908035 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc045bc-da0e-46f6-8298-da87f4ce83b8 service neutron] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:04.921693 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc045bc-da0e-46f6-8298-da87f4ce83b8 service neutron] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:05.057450 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c2ab7-4d9e-4329-9ea5-c57f91965e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:05.058559 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c2ab7-4d9e-4329-9ea5-c57f91965e6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:05.059083 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c2ab7-4d9e-4329-9ea5-c57f91965e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:05.059602 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c2ab7-4d9e-4329-9ea5-c57f91965e6a admin admin] PATH_INFO: `/v3/users/6d6b143b2c1640439ba4bc43e56a6118` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:05.066465 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c2ab7-4d9e-4329-9ea5-c57f91965e6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d6b143b2c1640439ba4bc43e56a6118)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:05.077399 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c2ab7-4d9e-4329-9ea5-c57f91965e6a admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:05.928306 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-1f8c2ab7-4d9e-4329-9ea5-c57f91965e6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d6b143b2c1640439ba4bc43e56a6118'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:05.952377 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-1f8c2ab7-4d9e-4329-9ea5-c57f91965e6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d6b143b2c1640439ba4bc43e56a6118'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:05.960263 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-1f8c2ab7-4d9e-4329-9ea5-c57f91965e6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d6b143b2c1640439ba4bc43e56a6118'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:06.120974 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-1f8c2ab7-4d9e-4329-9ea5-c57f91965e6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d6b143b2c1640439ba4bc43e56a6118'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:06.122469 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:06.153320 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-1f8c2ab7-4d9e-4329-9ea5-c57f91965e6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d6b143b2c1640439ba4bc43e56a6118'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:06.163339 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 876/1766] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:45:04 2022] DELETE /identity/v3/users/6d6b143b2c1640439ba4bc43e56a6118 => generated 0 bytes in 1256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:07.091861 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd53f96-58f8-4f54-8d13-2c00c877b622 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:07.099142 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-9fd53f96-58f8-4f54-8d13-2c00c877b622 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:07.176352 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd53f96-58f8-4f54-8d13-2c00c877b622 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:07.322883 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c2ab7-4d9e-4329-9ea5-c57f91965e6a admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:07.333414 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-1f8c2ab7-4d9e-4329-9ea5-c57f91965e6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:07.381888 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f45c51-3183-4231-aa5b-4266318604d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:07.383776 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f45c51-3183-4231-aa5b-4266318604d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:07.384750 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f45c51-3183-4231-aa5b-4266318604d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:07.386728 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f45c51-3183-4231-aa5b-4266318604d2 admin admin] PATH_INFO: `/v3/users/2facc43582f9418baf93abbedee45f44` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:07.391085 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f45c51-3183-4231-aa5b-4266318604d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2facc43582f9418baf93abbedee45f44)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:07.396224 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f45c51-3183-4231-aa5b-4266318604d2 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:07.548263 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c2ab7-4d9e-4329-9ea5-c57f91965e6a admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:07.558859 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750bcabf-2641-43aa-9b7b-8ed072516255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:07.562797 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750bcabf-2641-43aa-9b7b-8ed072516255 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:07.562797 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750bcabf-2641-43aa-9b7b-8ed072516255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:07.562797 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750bcabf-2641-43aa-9b7b-8ed072516255 admin admin] PATH_INFO: `/v3/projects/4d0da914d5a14f10b6b1a28256c5805b` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:07.578998 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750bcabf-2641-43aa-9b7b-8ed072516255 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d0da914d5a14f10b6b1a28256c5805b)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:07.584865 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750bcabf-2641-43aa-9b7b-8ed072516255 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:07.805892 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-a5f45c51-3183-4231-aa5b-4266318604d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2facc43582f9418baf93abbedee45f44'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:07.814409 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-750bcabf-2641-43aa-9b7b-8ed072516255 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d0da914d5a14f10b6b1a28256c5805b', 'request_id': 'req-750bcabf-2641-43aa-9b7b-8ed072516255'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:07.838794 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-750bcabf-2641-43aa-9b7b-8ed072516255 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d0da914d5a14f10b6b1a28256c5805b', 'request_id': 'req-750bcabf-2641-43aa-9b7b-8ed072516255'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:07.891532 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-a5f45c51-3183-4231-aa5b-4266318604d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2facc43582f9418baf93abbedee45f44'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:07.902736 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-a5f45c51-3183-4231-aa5b-4266318604d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2facc43582f9418baf93abbedee45f44'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:07.919612 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-a5f45c51-3183-4231-aa5b-4266318604d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2facc43582f9418baf93abbedee45f44'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:07.920496 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-a5f45c51-3183-4231-aa5b-4266318604d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2facc43582f9418baf93abbedee45f44'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:07.927023 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 891/1767] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:45:07 2022] DELETE /identity/v3/users/2facc43582f9418baf93abbedee45f44 => generated 0 bytes in 836 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:07.931746 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143d9a0-af55-45d7-946b-f5d5a7b417d6 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:07.931746 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143d9a0-af55-45d7-946b-f5d5a7b417d6 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:07.931746 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143d9a0-af55-45d7-946b-f5d5a7b417d6 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:07.931746 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 892/1768] 10.222.0.15 () {54 vars in 921 bytes} [Wed Aug 24 05:45:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:45:07.931746 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b3a233-75f9-4504-80c1-c0c167919146 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:07.932216 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b3a233-75f9-4504-80c1-c0c167919146 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:07.932216 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b3a233-75f9-4504-80c1-c0c167919146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:07.954929 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-a8b3a233-75f9-4504-80c1-c0c167919146 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:07.958284 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-a8b3a233-75f9-4504-80c1-c0c167919146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:07.999358 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-750bcabf-2641-43aa-9b7b-8ed072516255 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d0da914d5a14f10b6b1a28256c5805b', 'request_id': 'req-750bcabf-2641-43aa-9b7b-8ed072516255'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:08.014778 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-750bcabf-2641-43aa-9b7b-8ed072516255 admin admin] The token cache is being invalidate because project 4d0da914d5a14f10b6b1a28256c5805b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:45:08.015412 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-750bcabf-2641-43aa-9b7b-8ed072516255 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:08.018225 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-750bcabf-2641-43aa-9b7b-8ed072516255 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:08.024438 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 877/1769] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:45:07 2022] DELETE /identity/v3/projects/4d0da914d5a14f10b6b1a28256c5805b => generated 0 bytes in 699 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:45:08.024438 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c79a95-f845-42a8-bfab-de48fe1ea181 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:08.024438 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c79a95-f845-42a8-bfab-de48fe1ea181 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:08.024777 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c79a95-f845-42a8-bfab-de48fe1ea181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:08.057188 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 893/1770] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:08.062800 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e286bd72-eab2-4430-a895-3fb7bfa9f70c None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:08.063179 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e286bd72-eab2-4430-a895-3fb7bfa9f70c None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:08.063535 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e286bd72-eab2-4430-a895-3fb7bfa9f70c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:08.072872 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:08.073293 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-c2c79a95-f845-42a8-bfab-de48fe1ea181 None None] MFA Rules not processed for user `fbecffb800584c5786813accd061421b`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:08.075696 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-c2c79a95-f845-42a8-bfab-de48fe1ea181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:08.088042 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-e286bd72-eab2-4430-a895-3fb7bfa9f70c None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:08.089313 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-e286bd72-eab2-4430-a895-3fb7bfa9f70c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:08.100452 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 894/1771] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:08.100452 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e286bd72-eab2-4430-a895-3fb7bfa9f70c None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:08.100452 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-e286bd72-eab2-4430-a895-3fb7bfa9f70c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:08.169954 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 878/1772] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Aug 24 05:45:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:08.180508 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a2984-8b64-433d-bdce-881346b89328 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:08.180508 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a2984-8b64-433d-bdce-881346b89328 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:08.180508 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a2984-8b64-433d-bdce-881346b89328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:08.180849 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e286bd72-eab2-4430-a895-3fb7bfa9f70c None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:08.186169 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02984e-279d-4bad-a116-44dc5f50bedf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:08.197222 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02984e-279d-4bad-a116-44dc5f50bedf service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:08.197331 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02984e-279d-4bad-a116-44dc5f50bedf service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:08.197569 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02984e-279d-4bad-a116-44dc5f50bedf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:08.224875 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb02984e-279d-4bad-a116-44dc5f50bedf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:08.226507 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-ee1a2984-8b64-433d-bdce-881346b89328 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:08.227885 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-ee1a2984-8b64-433d-bdce-881346b89328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:08.228017 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb02984e-279d-4bad-a116-44dc5f50bedf service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:08.243484 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 895/1773] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:45:08.244641 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02984e-279d-4bad-a116-44dc5f50bedf service neutron] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:08.246159 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-eb02984e-279d-4bad-a116-44dc5f50bedf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:08.251623 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 879/1774] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:08.253546 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a2984-8b64-433d-bdce-881346b89328 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:08.272963 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a2984-8b64-433d-bdce-881346b89328 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:08.276571 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02984e-279d-4bad-a116-44dc5f50bedf service neutron] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:08.278766 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7289a3-f8c9-49bd-8d4c-30358b104803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:08.279395 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7289a3-f8c9-49bd-8d4c-30358b104803 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:08.279673 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7289a3-f8c9-49bd-8d4c-30358b104803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:08.279948 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7289a3-f8c9-49bd-8d4c-30358b104803 admin admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:08.280636 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7289a3-f8c9-49bd-8d4c-30358b104803 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:45:08.280939 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7289a3-f8c9-49bd-8d4c-30358b104803 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:08.282577 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7289a3-f8c9-49bd-8d4c-30358b104803 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:08.285231 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285cc1e0-ebeb-486a-b473-a18d9a8ef397 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:08.286513 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285cc1e0-ebeb-486a-b473-a18d9a8ef397 service nova] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:08.287205 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285cc1e0-ebeb-486a-b473-a18d9a8ef397 service nova] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:08.287868 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285cc1e0-ebeb-486a-b473-a18d9a8ef397 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:08.308630 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285cc1e0-ebeb-486a-b473-a18d9a8ef397 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:08.312681 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285cc1e0-ebeb-486a-b473-a18d9a8ef397 service nova] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:08.316008 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 880/1775] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:45:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:45:08.320865 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7289a3-f8c9-49bd-8d4c-30358b104803 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:08.341683 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7289a3-f8c9-49bd-8d4c-30358b104803 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:08.347683 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 896/1776] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:45:08.353061 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ef93f-f00d-4667-8c61-70b1efab5586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:08.367274 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ef93f-f00d-4667-8c61-70b1efab5586 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:08.367274 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ef93f-f00d-4667-8c61-70b1efab5586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:08.367274 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ef93f-f00d-4667-8c61-70b1efab5586 admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:08.367274 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6ef93f-f00d-4667-8c61-70b1efab5586 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:08.367274 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6ef93f-f00d-4667-8c61-70b1efab5586 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:08.561398 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 881/1777] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:45:08 2022] POST /identity/v3/projects => generated 357 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:08.566578 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285cc1e0-ebeb-486a-b473-a18d9a8ef397 service nova] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:08.579460 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285cc1e0-ebeb-486a-b473-a18d9a8ef397 service nova] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:08.646641 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:08.669057 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d83b495-e887-43b8-83dc-ffe7734fb62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:08.669797 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d83b495-e887-43b8-83dc-ffe7734fb62d admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:08.670095 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d83b495-e887-43b8-83dc-ffe7734fb62d admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:08.670395 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d83b495-e887-43b8-83dc-ffe7734fb62d admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:08.671797 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d83b495-e887-43b8-83dc-ffe7734fb62d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:08.677657 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d83b495-e887-43b8-83dc-ffe7734fb62d admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:08.818574 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ef93f-f00d-4667-8c61-70b1efab5586 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:08.820427 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-9f6ef93f-f00d-4667-8c61-70b1efab5586 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:08.856726 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ef93f-f00d-4667-8c61-70b1efab5586 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:08.870864 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 897/1778] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:45:08 2022] POST /identity/v3/users => generated 279 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:08.872260 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522011d-923d-4a42-bce0-190dd8fa5d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:08.873210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522011d-923d-4a42-bce0-190dd8fa5d14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:08.873591 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522011d-923d-4a42-bce0-190dd8fa5d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:08.873998 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522011d-923d-4a42-bce0-190dd8fa5d14 admin admin] PATH_INFO: `/v3/users/fd0e275e4a4043af8f57e118e6326bd3` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:08.877593 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d522011d-923d-4a42-bce0-190dd8fa5d14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd0e275e4a4043af8f57e118e6326bd3)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:08.878994 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d83b495-e887-43b8-83dc-ffe7734fb62d admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:08.881101 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d522011d-923d-4a42-bce0-190dd8fa5d14 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:08.905888 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d83b495-e887-43b8-83dc-ffe7734fb62d admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:08.923511 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf44e6c-0955-4f1d-a660-5020c4cb8eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:08.923511 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf44e6c-0955-4f1d-a660-5020c4cb8eb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:08.923511 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf44e6c-0955-4f1d-a660-5020c4cb8eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:08.923511 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf44e6c-0955-4f1d-a660-5020c4cb8eb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:08.924024 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf44e6c-0955-4f1d-a660-5020c4cb8eb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:08.924024 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf44e6c-0955-4f1d-a660-5020c4cb8eb5 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:08.983199 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 898/1779] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:08 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:08.989636 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf44e6c-0955-4f1d-a660-5020c4cb8eb5 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:08.999949 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf44e6c-0955-4f1d-a660-5020c4cb8eb5 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:09.006338 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd66503-6e9c-4c45-8651-465de933a757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:09.007350 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd66503-6e9c-4c45-8651-465de933a757 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:09.007830 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd66503-6e9c-4c45-8651-465de933a757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:09.008304 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd66503-6e9c-4c45-8651-465de933a757 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:09.009504 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd66503-6e9c-4c45-8651-465de933a757 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:09.012742 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd66503-6e9c-4c45-8651-465de933a757 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:09.045729 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 899/1780] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:08 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:09.050575 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd66503-6e9c-4c45-8651-465de933a757 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:09.063596 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd66503-6e9c-4c45-8651-465de933a757 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:09.071796 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e752af-45da-4f25-82df-04632d25895c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:09.072553 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e752af-45da-4f25-82df-04632d25895c admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:09.072838 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e752af-45da-4f25-82df-04632d25895c admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:09.073125 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e752af-45da-4f25-82df-04632d25895c admin admin] PATH_INFO: `/v3/projects/7fb3224a7adc4264b0a1e585d39a0c14/users/1cc9623a9d614091b3716fca1bf4a57d/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:09.093422 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e752af-45da-4f25-82df-04632d25895c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fb3224a7adc4264b0a1e585d39a0c14, user_id=1cc9623a9d614091b3716fca1bf4a57d, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:09.096211 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e752af-45da-4f25-82df-04632d25895c admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:09.236459 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 900/1781] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:09 2022] PUT /identity/v3/projects/7fb3224a7adc4264b0a1e585d39a0c14/users/1cc9623a9d614091b3716fca1bf4a57d/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:45:09.238497 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-d522011d-923d-4a42-bce0-190dd8fa5d14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd0e275e4a4043af8f57e118e6326bd3'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:09.242223 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e752af-45da-4f25-82df-04632d25895c admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:09.250350 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-d522011d-923d-4a42-bce0-190dd8fa5d14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd0e275e4a4043af8f57e118e6326bd3'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:09.250665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-d522011d-923d-4a42-bce0-190dd8fa5d14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd0e275e4a4043af8f57e118e6326bd3'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:09.255464 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-85e752af-45da-4f25-82df-04632d25895c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:09.341918 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e752af-45da-4f25-82df-04632d25895c admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:09.348132 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085a979f-9107-4f16-8c35-79e6e351990d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:09.349031 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a979f-9107-4f16-8c35-79e6e351990d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:09.349313 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a979f-9107-4f16-8c35-79e6e351990d admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:09.349582 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a979f-9107-4f16-8c35-79e6e351990d admin admin] PATH_INFO: `/v3/projects/cde6e2a745e542619f2ee4cde56f0e8a` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:09.350809 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085a979f-9107-4f16-8c35-79e6e351990d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cde6e2a745e542619f2ee4cde56f0e8a)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:09.353644 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085a979f-9107-4f16-8c35-79e6e351990d admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:09.355678 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-d522011d-923d-4a42-bce0-190dd8fa5d14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd0e275e4a4043af8f57e118e6326bd3'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:09.375948 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-d522011d-923d-4a42-bce0-190dd8fa5d14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd0e275e4a4043af8f57e118e6326bd3'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:09.386121 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 882/1782] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:45:08 2022] DELETE /identity/v3/users/fd0e275e4a4043af8f57e118e6326bd3 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:09.387430 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf1815b-2594-448e-aaad-6bb5ce1a26e3 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:09.387725 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf1815b-2594-448e-aaad-6bb5ce1a26e3 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:09.388005 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf1815b-2594-448e-aaad-6bb5ce1a26e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:09.421731 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-cbf1815b-2594-448e-aaad-6bb5ce1a26e3 None None] MFA Rules not processed for user `1cc9623a9d614091b3716fca1bf4a57d`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:09.428084 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-cbf1815b-2594-448e-aaad-6bb5ce1a26e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:09.490394 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 883/1783] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:09 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:09.491347 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf1815b-2594-448e-aaad-6bb5ce1a26e3 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:09.492703 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-cbf1815b-2594-448e-aaad-6bb5ce1a26e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:09.526350 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-085a979f-9107-4f16-8c35-79e6e351990d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cde6e2a745e542619f2ee4cde56f0e8a', 'request_id': 'req-085a979f-9107-4f16-8c35-79e6e351990d'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:09.532945 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-085a979f-9107-4f16-8c35-79e6e351990d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cde6e2a745e542619f2ee4cde56f0e8a', 'request_id': 'req-085a979f-9107-4f16-8c35-79e6e351990d'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:09.560180 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf1815b-2594-448e-aaad-6bb5ce1a26e3 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:09.622621 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4a454e-be99-4489-ad37-502d0e79d865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:09.623318 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a454e-be99-4489-ad37-502d0e79d865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:09.623601 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a454e-be99-4489-ad37-502d0e79d865 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:09.623880 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a454e-be99-4489-ad37-502d0e79d865 admin admin] PATH_INFO: `/v3/users/36e1111855114cd0aa7dce35d38a396f` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:09.625132 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4a454e-be99-4489-ad37-502d0e79d865 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36e1111855114cd0aa7dce35d38a396f)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:09.627146 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4a454e-be99-4489-ad37-502d0e79d865 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:09.652417 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-085a979f-9107-4f16-8c35-79e6e351990d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cde6e2a745e542619f2ee4cde56f0e8a', 'request_id': 'req-085a979f-9107-4f16-8c35-79e6e351990d'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:09.669599 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-085a979f-9107-4f16-8c35-79e6e351990d admin admin] The token cache is being invalidate because project cde6e2a745e542619f2ee4cde56f0e8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:45:09.669901 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-085a979f-9107-4f16-8c35-79e6e351990d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:09.670591 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-085a979f-9107-4f16-8c35-79e6e351990d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:09.672168 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 901/1784] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:45:09 2022] DELETE /identity/v3/projects/cde6e2a745e542619f2ee4cde56f0e8a => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:09.673559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f38c6-4a0a-44c7-a07f-bc70216b3cdc None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:09.673838 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f38c6-4a0a-44c7-a07f-bc70216b3cdc None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:09.674115 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f38c6-4a0a-44c7-a07f-bc70216b3cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:09.695441 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-253f38c6-4a0a-44c7-a07f-bc70216b3cdc None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:09.696743 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-253f38c6-4a0a-44c7-a07f-bc70216b3cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:09.707169 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 902/1785] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:09.708304 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f38c6-4a0a-44c7-a07f-bc70216b3cdc None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:09.716693 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-253f38c6-4a0a-44c7-a07f-bc70216b3cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:09.723465 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f38c6-4a0a-44c7-a07f-bc70216b3cdc None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:09.729572 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57856cef-4608-4ad0-bd8c-4ec2d364af61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:09.730210 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57856cef-4608-4ad0-bd8c-4ec2d364af61 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:09.730489 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57856cef-4608-4ad0-bd8c-4ec2d364af61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:09.730773 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57856cef-4608-4ad0-bd8c-4ec2d364af61 admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:09.731416 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57856cef-4608-4ad0-bd8c-4ec2d364af61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:09.733426 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57856cef-4608-4ad0-bd8c-4ec2d364af61 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:09.911028 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 903/1786] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:45:09 2022] POST /identity/v3/projects => generated 355 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:09.912182 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db296122-2b05-4316-9551-0ea385fec3ef None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:09.912498 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db296122-2b05-4316-9551-0ea385fec3ef None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:09.912845 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db296122-2b05-4316-9551-0ea385fec3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:09.932246 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-8d4a454e-be99-4489-ad37-502d0e79d865 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36e1111855114cd0aa7dce35d38a396f'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:09.934136 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-db296122-2b05-4316-9551-0ea385fec3ef None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:09.935376 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-db296122-2b05-4316-9551-0ea385fec3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:09.939520 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-8d4a454e-be99-4489-ad37-502d0e79d865 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36e1111855114cd0aa7dce35d38a396f'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:09.940229 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-8d4a454e-be99-4489-ad37-502d0e79d865 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36e1111855114cd0aa7dce35d38a396f'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:09.999637 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 904/1787] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:10.000724 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db296122-2b05-4316-9551-0ea385fec3ef None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:10.002036 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-db296122-2b05-4316-9551-0ea385fec3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:10.021870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-8d4a454e-be99-4489-ad37-502d0e79d865 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36e1111855114cd0aa7dce35d38a396f'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:10.035686 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-8d4a454e-be99-4489-ad37-502d0e79d865 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36e1111855114cd0aa7dce35d38a396f'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:10.045791 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 884/1788] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:45:09 2022] DELETE /identity/v3/users/36e1111855114cd0aa7dce35d38a396f => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:10.048755 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4a454e-be99-4489-ad37-502d0e79d865 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:10.049558 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-8d4a454e-be99-4489-ad37-502d0e79d865 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:10.074502 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db296122-2b05-4316-9551-0ea385fec3ef None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:10.125451 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4a454e-be99-4489-ad37-502d0e79d865 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:10.132925 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15d6227-cef9-485b-8e39-cfaf7bccaa16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:10.133691 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d6227-cef9-485b-8e39-cfaf7bccaa16 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:10.134036 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d6227-cef9-485b-8e39-cfaf7bccaa16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:10.134407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d6227-cef9-485b-8e39-cfaf7bccaa16 admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:10.135126 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15d6227-cef9-485b-8e39-cfaf7bccaa16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:10.138416 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15d6227-cef9-485b-8e39-cfaf7bccaa16 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:10.138595 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4802509-74bb-407a-9709-d6caa7ab5594 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:10.138981 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4802509-74bb-407a-9709-d6caa7ab5594 service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:10.139339 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4802509-74bb-407a-9709-d6caa7ab5594 service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:10.139691 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4802509-74bb-407a-9709-d6caa7ab5594 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:10.141780 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-c4802509-74bb-407a-9709-d6caa7ab5594 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:10.157688 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4802509-74bb-407a-9709-d6caa7ab5594 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:10.161137 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4802509-74bb-407a-9709-d6caa7ab5594 service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:10.172950 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 905/1789] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:45:10.174499 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583dadf3-ed57-413a-b395-0850b9aefa78 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:10.175587 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583dadf3-ed57-413a-b395-0850b9aefa78 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:10.175587 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583dadf3-ed57-413a-b395-0850b9aefa78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:10.201573 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-583dadf3-ed57-413a-b395-0850b9aefa78 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:10.202878 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-583dadf3-ed57-413a-b395-0850b9aefa78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:10.216436 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 906/1790] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:10.222237 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956dc719-4216-4e8f-afd7-79e6c1db8651 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:10.222237 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956dc719-4216-4e8f-afd7-79e6c1db8651 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:10.222237 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956dc719-4216-4e8f-afd7-79e6c1db8651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:10.229685 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 885/1791] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:45:10 2022] POST /identity/v3/users => generated 284 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:10.230956 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15d6227-cef9-485b-8e39-cfaf7bccaa16 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:10.241648 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-956dc719-4216-4e8f-afd7-79e6c1db8651 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:10.242905 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-956dc719-4216-4e8f-afd7-79e6c1db8651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:10.243487 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15d6227-cef9-485b-8e39-cfaf7bccaa16 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:10.252240 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:10.253016 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a47fa-3ca7-4b2c-95a6-b7f8a01132c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:10.254112 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a47fa-3ca7-4b2c-95a6-b7f8a01132c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:10.254573 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a47fa-3ca7-4b2c-95a6-b7f8a01132c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:10.254986 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a47fa-3ca7-4b2c-95a6-b7f8a01132c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:10.255634 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 907/1792] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:10.256041 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563a47fa-3ca7-4b2c-95a6-b7f8a01132c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:45:10.256516 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956dc719-4216-4e8f-afd7-79e6c1db8651 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:10.258122 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563a47fa-3ca7-4b2c-95a6-b7f8a01132c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:10.259533 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563a47fa-3ca7-4b2c-95a6-b7f8a01132c6 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:10.267223 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956dc719-4216-4e8f-afd7-79e6c1db8651 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:10.277348 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 886/1793] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:45:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:45:10.278908 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a47fa-3ca7-4b2c-95a6-b7f8a01132c6 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:10.282485 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a8f8f-eda1-40c6-b1e0-2fd5f1b6462e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:10.283257 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a8f8f-eda1-40c6-b1e0-2fd5f1b6462e admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:10.283555 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a8f8f-eda1-40c6-b1e0-2fd5f1b6462e admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:10.283833 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a8f8f-eda1-40c6-b1e0-2fd5f1b6462e admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:10.285267 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590a8f8f-eda1-40c6-b1e0-2fd5f1b6462e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:10.287015 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590a8f8f-eda1-40c6-b1e0-2fd5f1b6462e admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:10.296665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a47fa-3ca7-4b2c-95a6-b7f8a01132c6 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:10.306008 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 908/1794] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:10 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:10.306008 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a8f8f-eda1-40c6-b1e0-2fd5f1b6462e admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:10.308593 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a302d-39c2-4cdf-a011-52ab9ce66ef1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:10.309478 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a302d-39c2-4cdf-a011-52ab9ce66ef1 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:10.309925 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a302d-39c2-4cdf-a011-52ab9ce66ef1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:10.310548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a302d-39c2-4cdf-a011-52ab9ce66ef1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:10.320437 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a8f8f-eda1-40c6-b1e0-2fd5f1b6462e admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:10.323733 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8a302d-39c2-4cdf-a011-52ab9ce66ef1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:10.325703 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8a302d-39c2-4cdf-a011-52ab9ce66ef1 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:10.326543 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0526d822-34d1-49e5-80a9-71d9ea274ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:10.327248 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0526d822-34d1-49e5-80a9-71d9ea274ce9 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:10.327599 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0526d822-34d1-49e5-80a9-71d9ea274ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:10.327945 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0526d822-34d1-49e5-80a9-71d9ea274ce9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:10.328752 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0526d822-34d1-49e5-80a9-71d9ea274ce9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:10.331817 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0526d822-34d1-49e5-80a9-71d9ea274ce9 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:10.364596 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 887/1795] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:45:10.366010 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a302d-39c2-4cdf-a011-52ab9ce66ef1 service neutron] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:10.381597 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a302d-39c2-4cdf-a011-52ab9ce66ef1 service neutron] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:10.389144 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11460ca-2588-455d-b1e5-6eb7f417a3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:10.389835 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11460ca-2588-455d-b1e5-6eb7f417a3a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:10.390422 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11460ca-2588-455d-b1e5-6eb7f417a3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:10.390869 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11460ca-2588-455d-b1e5-6eb7f417a3a3 admin admin] PATH_INFO: `/v3/projects/f2122143ce924f7680452df87cad747d/users/bab41be5f8944d55b4cc3ba73c9fee24/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:10.412385 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11460ca-2588-455d-b1e5-6eb7f417a3a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2122143ce924f7680452df87cad747d, user_id=bab41be5f8944d55b4cc3ba73c9fee24, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:10.416020 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11460ca-2588-455d-b1e5-6eb7f417a3a3 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:10.495497 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 909/1796] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:45:10 2022] POST /identity/v3/projects => generated 369 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:10.502107 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0526d822-34d1-49e5-80a9-71d9ea274ce9 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:10.521838 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:10.530543 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 888/1797] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:10 2022] PUT /identity/v3/projects/f2122143ce924f7680452df87cad747d/users/bab41be5f8944d55b4cc3ba73c9fee24/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:10.536432 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11460ca-2588-455d-b1e5-6eb7f417a3a3 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:10.625606 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0526d822-34d1-49e5-80a9-71d9ea274ce9 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:10.689651 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11460ca-2588-455d-b1e5-6eb7f417a3a3 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:10.696637 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b85d6ff-6e11-4a80-9a81-afda3286c167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:10.697699 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85d6ff-6e11-4a80-9a81-afda3286c167 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:10.698048 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85d6ff-6e11-4a80-9a81-afda3286c167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:10.698414 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85d6ff-6e11-4a80-9a81-afda3286c167 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:10.699131 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b85d6ff-6e11-4a80-9a81-afda3286c167 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:10.701121 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b85d6ff-6e11-4a80-9a81-afda3286c167 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:10.703802 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d9ca7-98d5-48a5-89dc-96bc13506fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:10.704473 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d9ca7-98d5-48a5-89dc-96bc13506fb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:10.704757 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d9ca7-98d5-48a5-89dc-96bc13506fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:10.705041 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d9ca7-98d5-48a5-89dc-96bc13506fb5 admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:10.705665 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231d9ca7-98d5-48a5-89dc-96bc13506fb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:10.709382 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231d9ca7-98d5-48a5-89dc-96bc13506fb5 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:10.716170 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 889/1798] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:10 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:45:10.717471 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b85d6ff-6e11-4a80-9a81-afda3286c167 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:10.731507 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b85d6ff-6e11-4a80-9a81-afda3286c167 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:10.736932 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8690f256-1d22-4689-be46-8cd62904a058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:10.739718 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690f256-1d22-4689-be46-8cd62904a058 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:10.740023 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690f256-1d22-4689-be46-8cd62904a058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:10.740300 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690f256-1d22-4689-be46-8cd62904a058 admin admin] PATH_INFO: `/v3/projects/f2122143ce924f7680452df87cad747d/users/bab41be5f8944d55b4cc3ba73c9fee24/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:10.742583 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8690f256-1d22-4689-be46-8cd62904a058 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2122143ce924f7680452df87cad747d, user_id=bab41be5f8944d55b4cc3ba73c9fee24, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:10.745424 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8690f256-1d22-4689-be46-8cd62904a058 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:10.771463 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 890/1799] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:10 2022] PUT /identity/v3/projects/f2122143ce924f7680452df87cad747d/users/bab41be5f8944d55b4cc3ba73c9fee24/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:10.773482 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770841f1-c5a5-4869-9dce-ff47cc180918 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:10.773839 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770841f1-c5a5-4869-9dce-ff47cc180918 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:10.774171 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770841f1-c5a5-4869-9dce-ff47cc180918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:10.812156 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-770841f1-c5a5-4869-9dce-ff47cc180918 None None] MFA Rules not processed for user `bab41be5f8944d55b4cc3ba73c9fee24`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:10.813637 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-770841f1-c5a5-4869-9dce-ff47cc180918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:10.848808 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 910/1800] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:45:10 2022] POST /identity/v3/users => generated 285 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:45:10.853217 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d9ca7-98d5-48a5-89dc-96bc13506fb5 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:10.866250 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d9ca7-98d5-48a5-89dc-96bc13506fb5 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:10.912203 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 891/1801] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:10 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:10.926112 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770841f1-c5a5-4869-9dce-ff47cc180918 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:10.927573 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-770841f1-c5a5-4869-9dce-ff47cc180918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:10.950106 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b795c693-9caf-4bc0-9075-a40bd3d2d030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:10.960649 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795c693-9caf-4bc0-9075-a40bd3d2d030 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:10.960649 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795c693-9caf-4bc0-9075-a40bd3d2d030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:10.960649 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795c693-9caf-4bc0-9075-a40bd3d2d030 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:10.960649 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b795c693-9caf-4bc0-9075-a40bd3d2d030 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:10.960649 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b795c693-9caf-4bc0-9075-a40bd3d2d030 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:10.980485 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 911/1802] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:10 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:10.981659 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b795c693-9caf-4bc0-9075-a40bd3d2d030 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:10.988650 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-b795c693-9caf-4bc0-9075-a40bd3d2d030 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:11.006467 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b795c693-9caf-4bc0-9075-a40bd3d2d030 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:11.013119 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa39cd15-9e41-4bdc-ba91-858f5f055867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:11.014448 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39cd15-9e41-4bdc-ba91-858f5f055867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:11.014660 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39cd15-9e41-4bdc-ba91-858f5f055867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:11.015021 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39cd15-9e41-4bdc-ba91-858f5f055867 admin admin] PATH_INFO: `/v3/projects/1b6112a8e36041a8bd25229c86f3f316` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:11.016791 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa39cd15-9e41-4bdc-ba91-858f5f055867 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b6112a8e36041a8bd25229c86f3f316)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:11.018876 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa39cd15-9e41-4bdc-ba91-858f5f055867 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:11.020868 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770841f1-c5a5-4869-9dce-ff47cc180918 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:11.029466 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8533cf1-c83e-4a90-807b-0d259080c372 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:11.030685 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8533cf1-c83e-4a90-807b-0d259080c372 service cinder] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:11.030962 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8533cf1-c83e-4a90-807b-0d259080c372 service cinder] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:11.031352 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8533cf1-c83e-4a90-807b-0d259080c372 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:11.053862 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8533cf1-c83e-4a90-807b-0d259080c372 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:11.057117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8533cf1-c83e-4a90-807b-0d259080c372 service cinder] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:11.079752 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 892/1803] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:10 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:45:11.080899 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8533cf1-c83e-4a90-807b-0d259080c372 service cinder] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:11.099496 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8533cf1-c83e-4a90-807b-0d259080c372 service cinder] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:11.109404 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4e856-b59a-4c60-a709-548015a6d89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:11.110563 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4e856-b59a-4c60-a709-548015a6d89f admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:11.111196 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4e856-b59a-4c60-a709-548015a6d89f admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:11.111679 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4e856-b59a-4c60-a709-548015a6d89f admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:11.113290 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb4e856-b59a-4c60-a709-548015a6d89f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:11.117023 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb4e856-b59a-4c60-a709-548015a6d89f admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:11.146482 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 893/1804] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:11.155875 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4e856-b59a-4c60-a709-548015a6d89f admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:11.169051 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4e856-b59a-4c60-a709-548015a6d89f admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:11.171648 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-fa39cd15-9e41-4bdc-ba91-858f5f055867 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b6112a8e36041a8bd25229c86f3f316', 'request_id': 'req-fa39cd15-9e41-4bdc-ba91-858f5f055867'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:11.190550 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-fa39cd15-9e41-4bdc-ba91-858f5f055867 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b6112a8e36041a8bd25229c86f3f316', 'request_id': 'req-fa39cd15-9e41-4bdc-ba91-858f5f055867'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:11.243780 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13fbc0d-3e87-4832-aacd-7d249d866fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:11.244499 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fbc0d-3e87-4832-aacd-7d249d866fa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:11.244788 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fbc0d-3e87-4832-aacd-7d249d866fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:11.245071 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fbc0d-3e87-4832-aacd-7d249d866fa3 admin admin] PATH_INFO: `/v3/projects/d449049a2c79446e90bcfe601c7bac5b/users/46e4af7e21cf481d98e207c5738324ef/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:11.262433 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13fbc0d-3e87-4832-aacd-7d249d866fa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d449049a2c79446e90bcfe601c7bac5b, user_id=46e4af7e21cf481d98e207c5738324ef, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:11.265178 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13fbc0d-3e87-4832-aacd-7d249d866fa3 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:11.343301 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-fa39cd15-9e41-4bdc-ba91-858f5f055867 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b6112a8e36041a8bd25229c86f3f316', 'request_id': 'req-fa39cd15-9e41-4bdc-ba91-858f5f055867'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:11.354894 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-fa39cd15-9e41-4bdc-ba91-858f5f055867 admin admin] The token cache is being invalidate because project 1b6112a8e36041a8bd25229c86f3f316 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:45:11.355202 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-fa39cd15-9e41-4bdc-ba91-858f5f055867 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:11.357858 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-fa39cd15-9e41-4bdc-ba91-858f5f055867 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:11.362944 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 912/1805] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:45:10 2022] DELETE /identity/v3/projects/1b6112a8e36041a8bd25229c86f3f316 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:45:11.363828 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa39cd15-9e41-4bdc-ba91-858f5f055867 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:11.365350 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-fa39cd15-9e41-4bdc-ba91-858f5f055867 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:11.379905 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa39cd15-9e41-4bdc-ba91-858f5f055867 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:11.385606 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7010a3c3-12c8-44ff-9130-0c31976e4f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:11.386402 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010a3c3-12c8-44ff-9130-0c31976e4f64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:11.386973 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010a3c3-12c8-44ff-9130-0c31976e4f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:11.387264 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010a3c3-12c8-44ff-9130-0c31976e4f64 admin admin] PATH_INFO: `/v3/projects/a3ac47b8d1a04316be031edc82d57a7a` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:11.388852 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7010a3c3-12c8-44ff-9130-0c31976e4f64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3ac47b8d1a04316be031edc82d57a7a)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:11.391018 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7010a3c3-12c8-44ff-9130-0c31976e4f64 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:11.406147 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 894/1806] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:11 2022] PUT /identity/v3/projects/d449049a2c79446e90bcfe601c7bac5b/users/46e4af7e21cf481d98e207c5738324ef/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:11.407272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13fbc0d-3e87-4832-aacd-7d249d866fa3 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:11.422606 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-a13fbc0d-3e87-4832-aacd-7d249d866fa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:11.523055 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13fbc0d-3e87-4832-aacd-7d249d866fa3 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:11.528497 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e02918-fdce-4bf0-88af-2b45b220dbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:11.529291 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e02918-fdce-4bf0-88af-2b45b220dbab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:11.529595 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e02918-fdce-4bf0-88af-2b45b220dbab admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:11.529865 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e02918-fdce-4bf0-88af-2b45b220dbab admin admin] PATH_INFO: `/v3/users/4555f303a71a4566bdea2ceb382abba5` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:11.531077 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e02918-fdce-4bf0-88af-2b45b220dbab admin admin] RBAC: Authorizing `identity:delete_user(user_id=4555f303a71a4566bdea2ceb382abba5)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:11.533163 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e02918-fdce-4bf0-88af-2b45b220dbab admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:11.631353 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-7010a3c3-12c8-44ff-9130-0c31976e4f64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3ac47b8d1a04316be031edc82d57a7a', 'request_id': 'req-7010a3c3-12c8-44ff-9130-0c31976e4f64'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:11.655774 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-7010a3c3-12c8-44ff-9130-0c31976e4f64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3ac47b8d1a04316be031edc82d57a7a', 'request_id': 'req-7010a3c3-12c8-44ff-9130-0c31976e4f64'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:11.721247 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:11.743128 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-7010a3c3-12c8-44ff-9130-0c31976e4f64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3ac47b8d1a04316be031edc82d57a7a', 'request_id': 'req-7010a3c3-12c8-44ff-9130-0c31976e4f64'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:11.763780 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-7010a3c3-12c8-44ff-9130-0c31976e4f64 admin admin] The token cache is being invalidate because project a3ac47b8d1a04316be031edc82d57a7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:45:11.764566 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-7010a3c3-12c8-44ff-9130-0c31976e4f64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:11.765664 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-7010a3c3-12c8-44ff-9130-0c31976e4f64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:11.769443 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 913/1807] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:45:11 2022] DELETE /identity/v3/projects/a3ac47b8d1a04316be031edc82d57a7a => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:11.772249 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fe03e-5943-449e-b34d-aff8cd7669fa None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:11.773016 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fe03e-5943-449e-b34d-aff8cd7669fa None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:11.773726 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fe03e-5943-449e-b34d-aff8cd7669fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:11.825755 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-e9e02918-fdce-4bf0-88af-2b45b220dbab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4555f303a71a4566bdea2ceb382abba5'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:11.827688 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-a21fe03e-5943-449e-b34d-aff8cd7669fa None None] MFA Rules not processed for user `46e4af7e21cf481d98e207c5738324ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:11.830231 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-a21fe03e-5943-449e-b34d-aff8cd7669fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:11.838644 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-e9e02918-fdce-4bf0-88af-2b45b220dbab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4555f303a71a4566bdea2ceb382abba5'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:11.839785 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-e9e02918-fdce-4bf0-88af-2b45b220dbab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4555f303a71a4566bdea2ceb382abba5'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:11.944641 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 914/1808] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:11.945716 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21fe03e-5943-449e-b34d-aff8cd7669fa None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:11.947244 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-a21fe03e-5943-449e-b34d-aff8cd7669fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:11.976416 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-e9e02918-fdce-4bf0-88af-2b45b220dbab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4555f303a71a4566bdea2ceb382abba5'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:11.991263 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-e9e02918-fdce-4bf0-88af-2b45b220dbab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4555f303a71a4566bdea2ceb382abba5'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:12.013409 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21fe03e-5943-449e-b34d-aff8cd7669fa None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:12.013631 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 895/1809] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:45:11 2022] DELETE /identity/v3/users/4555f303a71a4566bdea2ceb382abba5 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:12.015389 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e02918-fdce-4bf0-88af-2b45b220dbab admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:12.021213 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-e9e02918-fdce-4bf0-88af-2b45b220dbab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:12.082609 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfee19b6-1833-45e0-b28f-67628c5cc2b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:12.083422 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee19b6-1833-45e0-b28f-67628c5cc2b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:12.083807 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee19b6-1833-45e0-b28f-67628c5cc2b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:12.084191 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee19b6-1833-45e0-b28f-67628c5cc2b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:12.086505 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-cfee19b6-1833-45e0-b28f-67628c5cc2b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:12.122231 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e02918-fdce-4bf0-88af-2b45b220dbab admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:12.130488 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbbf963-2878-45c0-870e-e80c1825cedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:12.131913 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbf963-2878-45c0-870e-e80c1825cedf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:12.132198 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbf963-2878-45c0-870e-e80c1825cedf admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:12.132503 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbf963-2878-45c0-870e-e80c1825cedf admin admin] PATH_INFO: `/v3/users/471f506156a94b9989f4e228164c0b34` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:12.134717 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbbf963-2878-45c0-870e-e80c1825cedf admin admin] RBAC: Authorizing `identity:delete_user(user_id=471f506156a94b9989f4e228164c0b34)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:12.139545 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbbf963-2878-45c0-870e-e80c1825cedf admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:12.156903 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfee19b6-1833-45e0-b28f-67628c5cc2b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:12.159067 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfee19b6-1833-45e0-b28f-67628c5cc2b2 service cinder] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:12.170874 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 915/1810] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:11 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:45:12.171891 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfee19b6-1833-45e0-b28f-67628c5cc2b2 service cinder] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:12.173445 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-cfee19b6-1833-45e0-b28f-67628c5cc2b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:12.196156 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfee19b6-1833-45e0-b28f-67628c5cc2b2 service cinder] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:12.202293 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704bc1f3-eced-4c06-92a0-a302076072ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:12.203155 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704bc1f3-eced-4c06-92a0-a302076072ed admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:12.203533 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704bc1f3-eced-4c06-92a0-a302076072ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:12.203910 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704bc1f3-eced-4c06-92a0-a302076072ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:12.204712 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704bc1f3-eced-4c06-92a0-a302076072ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:12.207229 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704bc1f3-eced-4c06-92a0-a302076072ed admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:12.409676 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 916/1811] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:45:12 2022] POST /identity/v3/projects => generated 367 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:12.411230 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704bc1f3-eced-4c06-92a0-a302076072ed admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:12.479952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-fbbbf963-2878-45c0-870e-e80c1825cedf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '471f506156a94b9989f4e228164c0b34'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:12.486622 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704bc1f3-eced-4c06-92a0-a302076072ed admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:12.490505 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-fbbbf963-2878-45c0-870e-e80c1825cedf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '471f506156a94b9989f4e228164c0b34'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:12.491263 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-fbbbf963-2878-45c0-870e-e80c1825cedf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '471f506156a94b9989f4e228164c0b34'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:12.492528 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd66b32-68ac-4d20-8e6e-2ded061212bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:12.493222 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd66b32-68ac-4d20-8e6e-2ded061212bc admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:12.493440 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd66b32-68ac-4d20-8e6e-2ded061212bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:12.493717 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd66b32-68ac-4d20-8e6e-2ded061212bc admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:12.494351 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd66b32-68ac-4d20-8e6e-2ded061212bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:12.497101 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd66b32-68ac-4d20-8e6e-2ded061212bc admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:12.617582 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:12.622286 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-fbbbf963-2878-45c0-870e-e80c1825cedf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '471f506156a94b9989f4e228164c0b34'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:12.636055 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 917/1812] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:45:12 2022] POST /identity/v3/users => generated 290 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:12.639054 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd66b32-68ac-4d20-8e6e-2ded061212bc admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:12.643369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-9dd66b32-68ac-4d20-8e6e-2ded061212bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:12.659423 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-fbbbf963-2878-45c0-870e-e80c1825cedf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '471f506156a94b9989f4e228164c0b34'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:12.662941 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd66b32-68ac-4d20-8e6e-2ded061212bc admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:12.670639 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3111b-b660-4744-82d3-703085bc129c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:12.670639 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3111b-b660-4744-82d3-703085bc129c admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:12.670639 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3111b-b660-4744-82d3-703085bc129c admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:12.670639 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3111b-b660-4744-82d3-703085bc129c admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:12.671178 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a3111b-b660-4744-82d3-703085bc129c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:12.673130 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a3111b-b660-4744-82d3-703085bc129c admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:12.688261 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 918/1813] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:12 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:12.693600 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3111b-b660-4744-82d3-703085bc129c admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:12.693780 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 896/1814] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:45:12 2022] DELETE /identity/v3/users/471f506156a94b9989f4e228164c0b34 => generated 0 bytes in 679 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:12.708165 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3111b-b660-4744-82d3-703085bc129c admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:12.810820 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee3da1-efd7-4ec1-b45c-4143426f1e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:12.811618 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee3da1-efd7-4ec1-b45c-4143426f1e58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:12.811910 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee3da1-efd7-4ec1-b45c-4143426f1e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:12.812197 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee3da1-efd7-4ec1-b45c-4143426f1e58 admin admin] PATH_INFO: `/v3/projects/7d39e67cce00462aa26d741f375472ce/users/4a37e78c50614652adfc7330de6daa5f/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:12.829024 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cee3da1-efd7-4ec1-b45c-4143426f1e58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d39e67cce00462aa26d741f375472ce, user_id=4a37e78c50614652adfc7330de6daa5f, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:12.832458 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cee3da1-efd7-4ec1-b45c-4143426f1e58 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:12.979330 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 919/1815] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:12 2022] PUT /identity/v3/projects/7d39e67cce00462aa26d741f375472ce/users/4a37e78c50614652adfc7330de6daa5f/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:12.981171 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbbf963-2878-45c0-870e-e80c1825cedf admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:13.000485 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbbf963-2878-45c0-870e-e80c1825cedf admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:13.099890 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506395a3-e3eb-4a21-98a9-c1678a70a9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:13.100723 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506395a3-e3eb-4a21-98a9-c1678a70a9a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:13.101178 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506395a3-e3eb-4a21-98a9-c1678a70a9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:13.101625 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506395a3-e3eb-4a21-98a9-c1678a70a9a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:13.102293 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506395a3-e3eb-4a21-98a9-c1678a70a9a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:13.103994 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506395a3-e3eb-4a21-98a9-c1678a70a9a0 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:13.115878 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 897/1816] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:12 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:13.120804 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee3da1-efd7-4ec1-b45c-4143426f1e58 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:13.132548 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee3da1-efd7-4ec1-b45c-4143426f1e58 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:13.137534 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d36c4b-2c80-488e-9e74-b4d21d1ac2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:13.138242 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d36c4b-2c80-488e-9e74-b4d21d1ac2cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:13.138496 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d36c4b-2c80-488e-9e74-b4d21d1ac2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:13.138770 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d36c4b-2c80-488e-9e74-b4d21d1ac2cc admin admin] PATH_INFO: `/v3/projects/7d39e67cce00462aa26d741f375472ce/users/4a37e78c50614652adfc7330de6daa5f/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:13.141112 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d36c4b-2c80-488e-9e74-b4d21d1ac2cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d39e67cce00462aa26d741f375472ce, user_id=4a37e78c50614652adfc7330de6daa5f, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:13.144802 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d36c4b-2c80-488e-9e74-b4d21d1ac2cc admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:13.156841 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 920/1817] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:13 2022] PUT /identity/v3/projects/7d39e67cce00462aa26d741f375472ce/users/4a37e78c50614652adfc7330de6daa5f/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:13.169845 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b52d7-5fe3-4137-876b-912e909d6a89 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:13.170129 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b52d7-5fe3-4137-876b-912e909d6a89 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:13.170403 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b52d7-5fe3-4137-876b-912e909d6a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:13.205254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-694b52d7-5fe3-4137-876b-912e909d6a89 None None] MFA Rules not processed for user `4a37e78c50614652adfc7330de6daa5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:13.206652 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-694b52d7-5fe3-4137-876b-912e909d6a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:13.282423 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 898/1818] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:13 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:13.290393 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d36c4b-2c80-488e-9e74-b4d21d1ac2cc admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:13.293219 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-92d36c4b-2c80-488e-9e74-b4d21d1ac2cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:13.295794 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b52d7-5fe3-4137-876b-912e909d6a89 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:13.297220 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-694b52d7-5fe3-4137-876b-912e909d6a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:13.360805 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b52d7-5fe3-4137-876b-912e909d6a89 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:13.365387 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0870539-ba78-4890-b3b1-a317144e504d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:13.366004 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0870539-ba78-4890-b3b1-a317144e504d service cinder] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:13.366289 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0870539-ba78-4890-b3b1-a317144e504d service cinder] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:13.366561 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0870539-ba78-4890-b3b1-a317144e504d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:13.381059 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0870539-ba78-4890-b3b1-a317144e504d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:13.382866 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0870539-ba78-4890-b3b1-a317144e504d service cinder] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:13.395155 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 899/1819] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:13 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:45:13.443315 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d36c4b-2c80-488e-9e74-b4d21d1ac2cc admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:13.449187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d371b0-1347-400a-a416-f9d8ef782d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:13.449865 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d371b0-1347-400a-a416-f9d8ef782d4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:13.450156 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d371b0-1347-400a-a416-f9d8ef782d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:13.450439 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d371b0-1347-400a-a416-f9d8ef782d4d admin admin] PATH_INFO: `/v3/projects/778c1fe92d774ea88732bc38f33d7b22` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:13.451636 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d371b0-1347-400a-a416-f9d8ef782d4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=778c1fe92d774ea88732bc38f33d7b22)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:13.453690 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d371b0-1347-400a-a416-f9d8ef782d4d admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:13.472259 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0870539-ba78-4890-b3b1-a317144e504d service cinder] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:13.474415 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-c0870539-ba78-4890-b3b1-a317144e504d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:13.493330 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0870539-ba78-4890-b3b1-a317144e504d service cinder] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:13.502510 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd27ae-7ffd-4a09-ba2d-3593c72812e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:13.508828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd27ae-7ffd-4a09-ba2d-3593c72812e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:13.509135 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd27ae-7ffd-4a09-ba2d-3593c72812e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:13.509422 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd27ae-7ffd-4a09-ba2d-3593c72812e4 admin admin] PATH_INFO: `/v3/projects/ca5768f1027149dfa11b76d8c1196ded` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:13.510983 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd27ae-7ffd-4a09-ba2d-3593c72812e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca5768f1027149dfa11b76d8c1196ded)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:13.513102 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd27ae-7ffd-4a09-ba2d-3593c72812e4 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:13.628324 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:13.632112 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-76d371b0-1347-400a-a416-f9d8ef782d4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '778c1fe92d774ea88732bc38f33d7b22', 'request_id': 'req-76d371b0-1347-400a-a416-f9d8ef782d4d'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:13.638999 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-76d371b0-1347-400a-a416-f9d8ef782d4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '778c1fe92d774ea88732bc38f33d7b22', 'request_id': 'req-76d371b0-1347-400a-a416-f9d8ef782d4d'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:13.646012 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-20dd27ae-7ffd-4a09-ba2d-3593c72812e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca5768f1027149dfa11b76d8c1196ded', 'request_id': 'req-20dd27ae-7ffd-4a09-ba2d-3593c72812e4'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:13.652146 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-20dd27ae-7ffd-4a09-ba2d-3593c72812e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca5768f1027149dfa11b76d8c1196ded', 'request_id': 'req-20dd27ae-7ffd-4a09-ba2d-3593c72812e4'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:13.772607 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-76d371b0-1347-400a-a416-f9d8ef782d4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '778c1fe92d774ea88732bc38f33d7b22', 'request_id': 'req-76d371b0-1347-400a-a416-f9d8ef782d4d'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:13.790441 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-76d371b0-1347-400a-a416-f9d8ef782d4d admin admin] The token cache is being invalidate because project 778c1fe92d774ea88732bc38f33d7b22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:45:13.791099 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-76d371b0-1347-400a-a416-f9d8ef782d4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:13.792119 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-76d371b0-1347-400a-a416-f9d8ef782d4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:13.797014 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 921/1820] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:45:13 2022] DELETE /identity/v3/projects/778c1fe92d774ea88732bc38f33d7b22 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:45:13.797014 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d371b0-1347-400a-a416-f9d8ef782d4d admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:13.797014 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-76d371b0-1347-400a-a416-f9d8ef782d4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:13.841515 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-20dd27ae-7ffd-4a09-ba2d-3593c72812e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca5768f1027149dfa11b76d8c1196ded', 'request_id': 'req-20dd27ae-7ffd-4a09-ba2d-3593c72812e4'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:13.859113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-20dd27ae-7ffd-4a09-ba2d-3593c72812e4 admin admin] The token cache is being invalidate because project ca5768f1027149dfa11b76d8c1196ded was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:45:13.859674 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-20dd27ae-7ffd-4a09-ba2d-3593c72812e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:13.860800 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-20dd27ae-7ffd-4a09-ba2d-3593c72812e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:13.862573 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 900/1821] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:45:13 2022] DELETE /identity/v3/projects/ca5768f1027149dfa11b76d8c1196ded => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:13.864893 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf44aa5-7f82-4534-86f2-2b5be058fba5 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:13.865385 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf44aa5-7f82-4534-86f2-2b5be058fba5 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:13.865665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf44aa5-7f82-4534-86f2-2b5be058fba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:13.868726 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d371b0-1347-400a-a416-f9d8ef782d4d admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:13.889922 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-9bf44aa5-7f82-4534-86f2-2b5be058fba5 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:13.891807 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-9bf44aa5-7f82-4534-86f2-2b5be058fba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:13.931300 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57c1741-7e92-46cd-bc5a-505841af98b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:13.931936 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c1741-7e92-46cd-bc5a-505841af98b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:13.932226 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c1741-7e92-46cd-bc5a-505841af98b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:13.932661 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c1741-7e92-46cd-bc5a-505841af98b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:13.934858 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-d57c1741-7e92-46cd-bc5a-505841af98b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:13.969773 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 901/1822] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:13.972220 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721e325-e776-4272-af57-df384ec31535 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:13.972681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721e325-e776-4272-af57-df384ec31535 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:13.972959 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721e325-e776-4272-af57-df384ec31535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:13.999739 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-3721e325-e776-4272-af57-df384ec31535 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:14.001556 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-3721e325-e776-4272-af57-df384ec31535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:14.014202 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 902/1823] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:14.016076 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d0eeb-0c20-44aa-8b4d-7a4a46bd712e None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:14.016580 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d0eeb-0c20-44aa-8b4d-7a4a46bd712e None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:14.017001 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d0eeb-0c20-44aa-8b4d-7a4a46bd712e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:14.042019 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57c1741-7e92-46cd-bc5a-505841af98b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:14.042724 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-9d4d0eeb-0c20-44aa-8b4d-7a4a46bd712e None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:14.043534 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57c1741-7e92-46cd-bc5a-505841af98b2 service cinder] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:14.043948 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-9d4d0eeb-0c20-44aa-8b4d-7a4a46bd712e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:14.055842 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 903/1824] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:14.055842 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85e131-6d2b-4a60-ad42-7dff1dd639a7 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:14.055842 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85e131-6d2b-4a60-ad42-7dff1dd639a7 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:14.056143 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 922/1825] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:13 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:45:14.056243 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85e131-6d2b-4a60-ad42-7dff1dd639a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:14.056974 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57c1741-7e92-46cd-bc5a-505841af98b2 service cinder] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:14.070339 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57c1741-7e92-46cd-bc5a-505841af98b2 service cinder] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:14.077460 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734c166f-7ee3-4060-8b91-5765a3b54481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:14.077460 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734c166f-7ee3-4060-8b91-5765a3b54481 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:14.077460 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734c166f-7ee3-4060-8b91-5765a3b54481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:14.078165 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734c166f-7ee3-4060-8b91-5765a3b54481 admin admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:14.078623 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734c166f-7ee3-4060-8b91-5765a3b54481 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:45:14.078997 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734c166f-7ee3-4060-8b91-5765a3b54481 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:14.079941 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-8c85e131-6d2b-4a60-ad42-7dff1dd639a7 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:14.081477 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734c166f-7ee3-4060-8b91-5765a3b54481 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:14.082440 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-8c85e131-6d2b-4a60-ad42-7dff1dd639a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:14.093436 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 904/1826] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:14.096442 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 923/1827] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:45:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:45:14.100620 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c85e131-6d2b-4a60-ad42-7dff1dd639a7 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:14.101732 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734c166f-7ee3-4060-8b91-5765a3b54481 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:14.116792 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734c166f-7ee3-4060-8b91-5765a3b54481 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:14.120093 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c85e131-6d2b-4a60-ad42-7dff1dd639a7 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:14.123139 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bcd6de-84e2-46ff-97d6-a90dbd01d72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:14.123886 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bcd6de-84e2-46ff-97d6-a90dbd01d72e admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:14.124883 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bcd6de-84e2-46ff-97d6-a90dbd01d72e admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:14.124883 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bcd6de-84e2-46ff-97d6-a90dbd01d72e admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:14.125407 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bcd6de-84e2-46ff-97d6-a90dbd01d72e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:14.128377 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bcd6de-84e2-46ff-97d6-a90dbd01d72e admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:14.133762 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b87fd98-9cd0-4394-9f5e-c5834b0505a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:14.134392 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87fd98-9cd0-4394-9f5e-c5834b0505a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:14.134673 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87fd98-9cd0-4394-9f5e-c5834b0505a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:14.134947 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87fd98-9cd0-4394-9f5e-c5834b0505a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:14.135606 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b87fd98-9cd0-4394-9f5e-c5834b0505a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:45:14.135916 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b87fd98-9cd0-4394-9f5e-c5834b0505a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:14.137579 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b87fd98-9cd0-4394-9f5e-c5834b0505a1 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:14.154331 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 905/1828] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:45:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:45:14.163496 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b87fd98-9cd0-4394-9f5e-c5834b0505a1 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:14.170855 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b87fd98-9cd0-4394-9f5e-c5834b0505a1 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:14.196571 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720aff4c-55cc-4b77-8d66-81e8d1393fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:14.197648 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720aff4c-55cc-4b77-8d66-81e8d1393fe9 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:14.198171 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720aff4c-55cc-4b77-8d66-81e8d1393fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:14.198643 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720aff4c-55cc-4b77-8d66-81e8d1393fe9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:14.200357 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720aff4c-55cc-4b77-8d66-81e8d1393fe9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:14.203767 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720aff4c-55cc-4b77-8d66-81e8d1393fe9 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:14.241023 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 924/1829] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:45:14 2022] POST /identity/v3/projects => generated 375 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:14.243127 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bcd6de-84e2-46ff-97d6-a90dbd01d72e admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:14.269742 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:14.315221 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bcd6de-84e2-46ff-97d6-a90dbd01d72e admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:14.321169 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72333f-fa00-404b-884a-f3ebfd885928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:14.321799 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72333f-fa00-404b-884a-f3ebfd885928 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:14.322082 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72333f-fa00-404b-884a-f3ebfd885928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:14.322362 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72333f-fa00-404b-884a-f3ebfd885928 admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:14.323090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a72333f-fa00-404b-884a-f3ebfd885928 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:14.325100 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a72333f-fa00-404b-884a-f3ebfd885928 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:14.395145 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 906/1830] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:45:14 2022] POST /identity/v3/projects => generated 377 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:14.399399 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720aff4c-55cc-4b77-8d66-81e8d1393fe9 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:14.492875 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720aff4c-55cc-4b77-8d66-81e8d1393fe9 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:14.500692 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 925/1831] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:45:14 2022] POST /identity/v3/users => generated 288 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:14.504710 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1575c0-33a8-4b3b-b83b-38d9a163d427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:14.505991 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1575c0-33a8-4b3b-b83b-38d9a163d427 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:14.506369 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1575c0-33a8-4b3b-b83b-38d9a163d427 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:14.506739 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1575c0-33a8-4b3b-b83b-38d9a163d427 admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:14.507270 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72333f-fa00-404b-884a-f3ebfd885928 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:14.507682 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1575c0-33a8-4b3b-b83b-38d9a163d427 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:14.510724 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1575c0-33a8-4b3b-b83b-38d9a163d427 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:14.527703 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=85,keystone:INSERT=4,keystone:DELETE=11 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:14.530565 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72333f-fa00-404b-884a-f3ebfd885928 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:14.541398 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818856c6-fc71-4e3c-88e7-ef748de5b737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:14.542542 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818856c6-fc71-4e3c-88e7-ef748de5b737 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:14.543178 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818856c6-fc71-4e3c-88e7-ef748de5b737 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:14.543708 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818856c6-fc71-4e3c-88e7-ef748de5b737 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:14.545291 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818856c6-fc71-4e3c-88e7-ef748de5b737 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:14.555932 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818856c6-fc71-4e3c-88e7-ef748de5b737 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:14.586595 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 926/1832] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:45:14.588179 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818856c6-fc71-4e3c-88e7-ef748de5b737 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:14.606276 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818856c6-fc71-4e3c-88e7-ef748de5b737 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:14.614224 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b412855c-0a51-48f2-afb7-93dadbee737f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:14.614895 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412855c-0a51-48f2-afb7-93dadbee737f admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:14.615174 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412855c-0a51-48f2-afb7-93dadbee737f admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:14.615454 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412855c-0a51-48f2-afb7-93dadbee737f admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:14.616277 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b412855c-0a51-48f2-afb7-93dadbee737f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:14.617979 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b412855c-0a51-48f2-afb7-93dadbee737f admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:14.642659 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 927/1833] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:14.646795 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b412855c-0a51-48f2-afb7-93dadbee737f admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:14.657259 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b412855c-0a51-48f2-afb7-93dadbee737f admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:14.663268 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02007a58-f946-4645-8787-165b9d6ae822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:14.664246 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02007a58-f946-4645-8787-165b9d6ae822 admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:14.664587 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02007a58-f946-4645-8787-165b9d6ae822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:14.664863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02007a58-f946-4645-8787-165b9d6ae822 admin admin] PATH_INFO: `/v3/projects/6cf816bd33084fb5b3c9402edcb65788/users/ec7efa9e4ce74f2484d39779d033df3e/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:14.692232 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02007a58-f946-4645-8787-165b9d6ae822 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cf816bd33084fb5b3c9402edcb65788, user_id=ec7efa9e4ce74f2484d39779d033df3e, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:14.695418 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02007a58-f946-4645-8787-165b9d6ae822 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:14.709861 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 907/1834] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:45:14 2022] POST /identity/v3/users => generated 289 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:45:14.714791 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1575c0-33a8-4b3b-b83b-38d9a163d427 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:14.741202 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1575c0-33a8-4b3b-b83b-38d9a163d427 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:14.748357 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff62e3a-a366-4fd7-86d0-5aa6ba0a08a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:14.749106 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff62e3a-a366-4fd7-86d0-5aa6ba0a08a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:14.749392 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff62e3a-a366-4fd7-86d0-5aa6ba0a08a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:14.749672 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff62e3a-a366-4fd7-86d0-5aa6ba0a08a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:14.750364 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff62e3a-a366-4fd7-86d0-5aa6ba0a08a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:14.752150 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff62e3a-a366-4fd7-86d0-5aa6ba0a08a2 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:14.765474 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 908/1835] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:14.771975 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff62e3a-a366-4fd7-86d0-5aa6ba0a08a2 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:14.782972 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff62e3a-a366-4fd7-86d0-5aa6ba0a08a2 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:14.788660 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16aa07-9e4c-40e7-9eac-7359b8969e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:14.789285 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16aa07-9e4c-40e7-9eac-7359b8969e98 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:14.789568 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16aa07-9e4c-40e7-9eac-7359b8969e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:14.789848 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16aa07-9e4c-40e7-9eac-7359b8969e98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:14.790832 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f16aa07-9e4c-40e7-9eac-7359b8969e98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:14.792493 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f16aa07-9e4c-40e7-9eac-7359b8969e98 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:14.807435 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 909/1836] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:45:14.815529 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16aa07-9e4c-40e7-9eac-7359b8969e98 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:14.815730 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 928/1837] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:14 2022] PUT /identity/v3/projects/6cf816bd33084fb5b3c9402edcb65788/users/ec7efa9e4ce74f2484d39779d033df3e/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:45:14.819717 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16aa07-9e4c-40e7-9eac-7359b8969e98 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:14.825075 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d4a4e7-dd0b-4809-baf3-3065ee4012af None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:14.825368 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d4a4e7-dd0b-4809-baf3-3065ee4012af None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:14.825750 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d4a4e7-dd0b-4809-baf3-3065ee4012af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:14.855712 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-b6d4a4e7-dd0b-4809-baf3-3065ee4012af None None] MFA Rules not processed for user `ec7efa9e4ce74f2484d39779d033df3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:14.857017 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-b6d4a4e7-dd0b-4809-baf3-3065ee4012af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:14.893912 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb05e7-5335-46af-a1b3-1fdb904978a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:14.894406 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb05e7-5335-46af-a1b3-1fdb904978a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:14.894685 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb05e7-5335-46af-a1b3-1fdb904978a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:14.894955 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb05e7-5335-46af-a1b3-1fdb904978a2 admin admin] PATH_INFO: `/v3/projects/3f94937573cf4ee3907e3331c171df84/users/4b9a15110190443c8950eb6861196205/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:14.911287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fb05e7-5335-46af-a1b3-1fdb904978a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f94937573cf4ee3907e3331c171df84, user_id=4b9a15110190443c8950eb6861196205, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:14.914741 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fb05e7-5335-46af-a1b3-1fdb904978a2 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:14.938081 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 929/1838] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:45:15.006573 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 910/1839] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:14 2022] PUT /identity/v3/projects/3f94937573cf4ee3907e3331c171df84/users/4b9a15110190443c8950eb6861196205/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:15.017134 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29199f6-1393-48a6-8a76-43196d2bf6ac None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:15.017501 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29199f6-1393-48a6-8a76-43196d2bf6ac None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:15.017789 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29199f6-1393-48a6-8a76-43196d2bf6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:15.033707 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:15.050887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-d29199f6-1393-48a6-8a76-43196d2bf6ac None None] MFA Rules not processed for user `4b9a15110190443c8950eb6861196205`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:15.052139 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-d29199f6-1393-48a6-8a76-43196d2bf6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:15.072160 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb05e7-5335-46af-a1b3-1fdb904978a2 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:15.075337 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-12fb05e7-5335-46af-a1b3-1fdb904978a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:15.139219 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 930/1840] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:15.149533 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb05e7-5335-46af-a1b3-1fdb904978a2 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:15.149781 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29199f6-1393-48a6-8a76-43196d2bf6ac None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:15.154370 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb5a60-6ccc-4ed1-90b5-8df437fae4f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:15.155155 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb5a60-6ccc-4ed1-90b5-8df437fae4f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:15.155442 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb5a60-6ccc-4ed1-90b5-8df437fae4f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:15.160501 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb5a60-6ccc-4ed1-90b5-8df437fae4f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:15.259381 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29199f6-1393-48a6-8a76-43196d2bf6ac None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:15.263968 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceb5a60-6ccc-4ed1-90b5-8df437fae4f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:15.265555 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceb5a60-6ccc-4ed1-90b5-8df437fae4f8 service cinder] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:15.269083 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0994a7c9-1343-4f8c-ad12-4cffce647f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:15.270435 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994a7c9-1343-4f8c-ad12-4cffce647f89 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:15.272174 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994a7c9-1343-4f8c-ad12-4cffce647f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:15.272690 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994a7c9-1343-4f8c-ad12-4cffce647f89 admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:15.274147 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0994a7c9-1343-4f8c-ad12-4cffce647f89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:15.276660 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 911/1841] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:45:15.277679 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0994a7c9-1343-4f8c-ad12-4cffce647f89 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:15.412255 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb5a60-6ccc-4ed1-90b5-8df437fae4f8 service cinder] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:15.419762 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 931/1842] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:45:15 2022] POST /identity/v3/projects => generated 379 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:15.426463 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb5a60-6ccc-4ed1-90b5-8df437fae4f8 service cinder] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:15.541860 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d6897-751d-45ca-9447-e4afaffea397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:15.542870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d6897-751d-45ca-9447-e4afaffea397 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:15.543350 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d6897-751d-45ca-9447-e4afaffea397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:15.543792 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d6897-751d-45ca-9447-e4afaffea397 admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:15.544880 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9d6897-751d-45ca-9447-e4afaffea397 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:15.548709 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9d6897-751d-45ca-9447-e4afaffea397 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:15.736808 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:15.751990 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 912/1843] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:45:15 2022] POST /identity/v3/users => generated 290 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:15.752312 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0994a7c9-1343-4f8c-ad12-4cffce647f89 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:15.770490 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0994a7c9-1343-4f8c-ad12-4cffce647f89 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:15.780124 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5613c11-87df-47a6-886e-cb271b8af8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:15.780467 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5613c11-87df-47a6-886e-cb271b8af8a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:15.780775 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5613c11-87df-47a6-886e-cb271b8af8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:15.781055 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5613c11-87df-47a6-886e-cb271b8af8a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:15.781726 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5613c11-87df-47a6-886e-cb271b8af8a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:15.783690 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5613c11-87df-47a6-886e-cb271b8af8a7 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:15.799147 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 932/1844] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:45:15.800562 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d6897-751d-45ca-9447-e4afaffea397 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:15.815349 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d6897-751d-45ca-9447-e4afaffea397 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:15.825855 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd43b479-8121-4226-a04f-0e193b9c7088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:15.825855 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd43b479-8121-4226-a04f-0e193b9c7088 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:15.825855 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd43b479-8121-4226-a04f-0e193b9c7088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:15.825855 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd43b479-8121-4226-a04f-0e193b9c7088 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:15.826228 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd43b479-8121-4226-a04f-0e193b9c7088 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:15.826228 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd43b479-8121-4226-a04f-0e193b9c7088 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:15.848182 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 913/1845] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:15.858052 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5613c11-87df-47a6-886e-cb271b8af8a7 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:15.872268 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5613c11-87df-47a6-886e-cb271b8af8a7 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:15.879379 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b882cb4f-241f-4e6f-8376-262d0f6ec2c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:15.880081 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882cb4f-241f-4e6f-8376-262d0f6ec2c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:15.880363 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882cb4f-241f-4e6f-8376-262d0f6ec2c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:15.880695 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882cb4f-241f-4e6f-8376-262d0f6ec2c1 admin admin] PATH_INFO: `/v3/projects/5ddc63f6f46e455780ace9ab6a871913/users/d2a559bf8f604af89a26c760799939a0/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:15.901055 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b882cb4f-241f-4e6f-8376-262d0f6ec2c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ddc63f6f46e455780ace9ab6a871913, user_id=d2a559bf8f604af89a26c760799939a0, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:15.904238 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b882cb4f-241f-4e6f-8376-262d0f6ec2c1 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:16.058213 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 933/1846] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:15 2022] PUT /identity/v3/projects/5ddc63f6f46e455780ace9ab6a871913/users/d2a559bf8f604af89a26c760799939a0/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:45:16.069948 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed1b23-0c17-488d-bc85-5661a36adfc0 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:16.070235 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed1b23-0c17-488d-bc85-5661a36adfc0 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:16.070519 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed1b23-0c17-488d-bc85-5661a36adfc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:16.102027 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-b4ed1b23-0c17-488d-bc85-5661a36adfc0 None None] MFA Rules not processed for user `d2a559bf8f604af89a26c760799939a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:16.103262 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-b4ed1b23-0c17-488d-bc85-5661a36adfc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:16.172761 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 914/1847] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:16.178845 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b882cb4f-241f-4e6f-8376-262d0f6ec2c1 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:16.285408 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b882cb4f-241f-4e6f-8376-262d0f6ec2c1 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:16.291861 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68f1177-1162-4833-8b18-776114eafa61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:16.292523 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68f1177-1162-4833-8b18-776114eafa61 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:16.292808 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68f1177-1162-4833-8b18-776114eafa61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:16.293084 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68f1177-1162-4833-8b18-776114eafa61 admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:16.293743 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68f1177-1162-4833-8b18-776114eafa61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:16.295681 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68f1177-1162-4833-8b18-776114eafa61 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:16.403578 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 934/1848] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:45:16 2022] POST /identity/v3/projects => generated 379 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:45:16.408145 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed1b23-0c17-488d-bc85-5661a36adfc0 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:16.419733 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed1b23-0c17-488d-bc85-5661a36adfc0 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:16.478092 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2509ab2b-0edd-42a9-847a-82603b90e879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:16.478780 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509ab2b-0edd-42a9-847a-82603b90e879 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:16.479133 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509ab2b-0edd-42a9-847a-82603b90e879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:16.479404 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509ab2b-0edd-42a9-847a-82603b90e879 admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:16.480036 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2509ab2b-0edd-42a9-847a-82603b90e879 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:16.481974 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2509ab2b-0edd-42a9-847a-82603b90e879 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:16.640864 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 915/1849] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:45:16 2022] POST /identity/v3/users => generated 296 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:16.645969 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68f1177-1162-4833-8b18-776114eafa61 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:16.666844 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68f1177-1162-4833-8b18-776114eafa61 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:16.672312 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797caeee-3710-42c7-b637-e4291b3fc8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:16.672974 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797caeee-3710-42c7-b637-e4291b3fc8ce admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:16.673304 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797caeee-3710-42c7-b637-e4291b3fc8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:16.673582 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797caeee-3710-42c7-b637-e4291b3fc8ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:16.674264 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797caeee-3710-42c7-b637-e4291b3fc8ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:16.675932 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797caeee-3710-42c7-b637-e4291b3fc8ce admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:16.692276 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 935/1850] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:16 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:16.697213 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2509ab2b-0edd-42a9-847a-82603b90e879 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:16.715464 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2509ab2b-0edd-42a9-847a-82603b90e879 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:16.728167 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c3d909-6a71-46f7-a1af-d71394769c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:16.729013 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3d909-6a71-46f7-a1af-d71394769c3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:16.729436 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3d909-6a71-46f7-a1af-d71394769c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:16.729791 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3d909-6a71-46f7-a1af-d71394769c3f admin admin] PATH_INFO: `/v3/projects/502817dabbac489997a3c7de0dcebc69/users/c378ee0a0139443da7462ed19adb46f4/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:16.753946 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c3d909-6a71-46f7-a1af-d71394769c3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=502817dabbac489997a3c7de0dcebc69, user_id=c378ee0a0139443da7462ed19adb46f4, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:16.757002 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c3d909-6a71-46f7-a1af-d71394769c3f admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:16.866265 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 916/1851] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:16 2022] PUT /identity/v3/projects/502817dabbac489997a3c7de0dcebc69/users/c378ee0a0139443da7462ed19adb46f4/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:45:16.871576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797caeee-3710-42c7-b637-e4291b3fc8ce admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:16.885537 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797caeee-3710-42c7-b637-e4291b3fc8ce admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:16.903728 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:16.974502 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e287c77-01a7-42c9-ad63-f3c9311fec63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:16.975224 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e287c77-01a7-42c9-ad63-f3c9311fec63 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:16.975666 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e287c77-01a7-42c9-ad63-f3c9311fec63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:16.975875 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e287c77-01a7-42c9-ad63-f3c9311fec63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:16.976593 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e287c77-01a7-42c9-ad63-f3c9311fec63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:16.978183 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e287c77-01a7-42c9-ad63-f3c9311fec63 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:16.994083 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 936/1852] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:16 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:16.997982 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c3d909-6a71-46f7-a1af-d71394769c3f admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:17.011689 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c3d909-6a71-46f7-a1af-d71394769c3f admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:17.018053 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6590863b-afc8-47eb-8937-c516540d73aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:17.025869 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6590863b-afc8-47eb-8937-c516540d73aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:17.025869 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6590863b-afc8-47eb-8937-c516540d73aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:17.025869 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6590863b-afc8-47eb-8937-c516540d73aa admin admin] PATH_INFO: `/v3/projects/502817dabbac489997a3c7de0dcebc69/users/c378ee0a0139443da7462ed19adb46f4/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:17.025869 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6590863b-afc8-47eb-8937-c516540d73aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=502817dabbac489997a3c7de0dcebc69, user_id=c378ee0a0139443da7462ed19adb46f4, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:17.025869 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6590863b-afc8-47eb-8937-c516540d73aa admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:17.036786 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 917/1853] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:16 2022] PUT /identity/v3/projects/502817dabbac489997a3c7de0dcebc69/users/c378ee0a0139443da7462ed19adb46f4/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:17.049384 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076cc403-928a-4c42-b941-c5e932a6328d None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:17.049894 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076cc403-928a-4c42-b941-c5e932a6328d None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:17.060724 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076cc403-928a-4c42-b941-c5e932a6328d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:17.084136 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-076cc403-928a-4c42-b941-c5e932a6328d None None] MFA Rules not processed for user `c378ee0a0139443da7462ed19adb46f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:17.085508 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-076cc403-928a-4c42-b941-c5e932a6328d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:17.184744 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 937/1854] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:17 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:17.566891 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6590863b-afc8-47eb-8937-c516540d73aa admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:17.568953 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-6590863b-afc8-47eb-8937-c516540d73aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:17.641907 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6590863b-afc8-47eb-8937-c516540d73aa admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:17.646391 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133ab759-2fbc-4d48-a94c-c01aa8d257c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:17.647013 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133ab759-2fbc-4d48-a94c-c01aa8d257c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:17.647299 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133ab759-2fbc-4d48-a94c-c01aa8d257c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:17.647576 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133ab759-2fbc-4d48-a94c-c01aa8d257c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:17.657887 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133ab759-2fbc-4d48-a94c-c01aa8d257c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:17.659696 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133ab759-2fbc-4d48-a94c-c01aa8d257c4 service cinder] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:17.670394 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 918/1855] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:17 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:45:17.923084 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076cc403-928a-4c42-b941-c5e932a6328d None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:17.934261 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:17.943715 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076cc403-928a-4c42-b941-c5e932a6328d None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:17.977788 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7962ae-3827-4e48-a8a2-d844ae039ed1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:17.981253 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7962ae-3827-4e48-a8a2-d844ae039ed1 service cinder] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:17.981680 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7962ae-3827-4e48-a8a2-d844ae039ed1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:17.983671 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7962ae-3827-4e48-a8a2-d844ae039ed1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:18.104211 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7962ae-3827-4e48-a8a2-d844ae039ed1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:18.105989 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7962ae-3827-4e48-a8a2-d844ae039ed1 service cinder] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:18.118921 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 938/1856] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:45:19.326442 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:19.616975 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:20.300169 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:20.376453 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf5e69-fca1-49e2-a29a-85da015bd768 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:20.378800 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf5e69-fca1-49e2-a29a-85da015bd768 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:20.378800 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf5e69-fca1-49e2-a29a-85da015bd768 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:20.379843 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 919/1857] 10.222.0.15 () {54 vars in 921 bytes} [Wed Aug 24 05:45:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:45:20.402278 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f94afdf-5474-4ae7-8bfc-934c54df5346 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:20.402716 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f94afdf-5474-4ae7-8bfc-934c54df5346 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:20.403146 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f94afdf-5474-4ae7-8bfc-934c54df5346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:20.448845 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-6f94afdf-5474-4ae7-8bfc-934c54df5346 None None] MFA Rules not processed for user `fbecffb800584c5786813accd061421b`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:20.448845 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-6f94afdf-5474-4ae7-8bfc-934c54df5346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:20.543719 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 939/1858] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Aug 24 05:45:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:45:20.561809 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbf5e69-fca1-49e2-a29a-85da015bd768 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:20.563615 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-9cbf5e69-fca1-49e2-a29a-85da015bd768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:20.585295 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbf5e69-fca1-49e2-a29a-85da015bd768 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:20.589956 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52b16ff-f3a5-466a-88fb-357ef7cfcb39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:20.590578 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b16ff-f3a5-466a-88fb-357ef7cfcb39 service nova] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:20.590863 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b16ff-f3a5-466a-88fb-357ef7cfcb39 service nova] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:20.591148 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b16ff-f3a5-466a-88fb-357ef7cfcb39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:20.600712 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52b16ff-f3a5-466a-88fb-357ef7cfcb39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:20.602309 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52b16ff-f3a5-466a-88fb-357ef7cfcb39 service nova] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:20.613016 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 920/1859] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:45:21.561379 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:21.983475 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f94afdf-5474-4ae7-8bfc-934c54df5346 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:21.985060 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-6f94afdf-5474-4ae7-8bfc-934c54df5346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:22.099706 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f94afdf-5474-4ae7-8bfc-934c54df5346 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:22.108827 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e90536-d70f-4b04-8e94-f5da9402fcff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:22.109675 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e90536-d70f-4b04-8e94-f5da9402fcff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:22.113413 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e90536-d70f-4b04-8e94-f5da9402fcff admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:22.113413 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e90536-d70f-4b04-8e94-f5da9402fcff admin admin] PATH_INFO: `/v3/users/46e4af7e21cf481d98e207c5738324ef` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:22.113413 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e90536-d70f-4b04-8e94-f5da9402fcff admin admin] RBAC: Authorizing `identity:delete_user(user_id=46e4af7e21cf481d98e207c5738324ef)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:22.115228 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e90536-d70f-4b04-8e94-f5da9402fcff admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:22.217212 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52b16ff-f3a5-466a-88fb-357ef7cfcb39 service nova] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:22.231860 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52b16ff-f3a5-466a-88fb-357ef7cfcb39 service nova] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:22.245092 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3105ed91-5ef8-475d-9b35-3bdd2591924b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:22.247287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105ed91-5ef8-475d-9b35-3bdd2591924b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:22.247640 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105ed91-5ef8-475d-9b35-3bdd2591924b admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:22.248141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105ed91-5ef8-475d-9b35-3bdd2591924b admin admin] PATH_INFO: `/v3/users/ec7efa9e4ce74f2484d39779d033df3e` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:22.249779 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3105ed91-5ef8-475d-9b35-3bdd2591924b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec7efa9e4ce74f2484d39779d033df3e)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:22.252838 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3105ed91-5ef8-475d-9b35-3bdd2591924b admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:22.432758 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-98e90536-d70f-4b04-8e94-f5da9402fcff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46e4af7e21cf481d98e207c5738324ef'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:22.578747 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-98e90536-d70f-4b04-8e94-f5da9402fcff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46e4af7e21cf481d98e207c5738324ef'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:22.594507 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-3105ed91-5ef8-475d-9b35-3bdd2591924b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec7efa9e4ce74f2484d39779d033df3e'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:22.602719 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-3105ed91-5ef8-475d-9b35-3bdd2591924b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec7efa9e4ce74f2484d39779d033df3e'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:22.603474 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-3105ed91-5ef8-475d-9b35-3bdd2591924b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec7efa9e4ce74f2484d39779d033df3e'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:22.612868 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-98e90536-d70f-4b04-8e94-f5da9402fcff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46e4af7e21cf481d98e207c5738324ef'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:22.629363 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-98e90536-d70f-4b04-8e94-f5da9402fcff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46e4af7e21cf481d98e207c5738324ef'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:22.630435 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-98e90536-d70f-4b04-8e94-f5da9402fcff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46e4af7e21cf481d98e207c5738324ef'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:22.640855 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 940/1860] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:45:21 2022] DELETE /identity/v3/users/46e4af7e21cf481d98e207c5738324ef => generated 0 bytes in 658 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:22.648804 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fdbe9-4a15-47f3-878f-583515038b03 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:22.649100 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fdbe9-4a15-47f3-878f-583515038b03 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:22.649409 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fdbe9-4a15-47f3-878f-583515038b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:22.682440 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-af6fdbe9-4a15-47f3-878f-583515038b03 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:22.685061 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-af6fdbe9-4a15-47f3-878f-583515038b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:22.811318 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-3105ed91-5ef8-475d-9b35-3bdd2591924b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec7efa9e4ce74f2484d39779d033df3e'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:22.813094 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 941/1861] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:45:22.829012 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6fdbe9-4a15-47f3-878f-583515038b03 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:22.831412 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-af6fdbe9-4a15-47f3-878f-583515038b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:22.835920 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:22.853942 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-3105ed91-5ef8-475d-9b35-3bdd2591924b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec7efa9e4ce74f2484d39779d033df3e'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:22.880102 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 921/1862] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:45:22 2022] DELETE /identity/v3/users/ec7efa9e4ce74f2484d39779d033df3e => generated 0 bytes in 664 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:45:22.886214 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c1591-91b2-4ad3-9538-e23c23576e6a None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:22.886732 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c1591-91b2-4ad3-9538-e23c23576e6a None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:22.887221 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c1591-91b2-4ad3-9538-e23c23576e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:22.918224 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-dc6c1591-91b2-4ad3-9538-e23c23576e6a None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:22.919292 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-dc6c1591-91b2-4ad3-9538-e23c23576e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:22.928029 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6fdbe9-4a15-47f3-878f-583515038b03 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:22.957669 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:23.009422 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c36d58-aa3e-4826-b667-9d2e68364a03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:23.010138 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c36d58-aa3e-4826-b667-9d2e68364a03 service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:23.010582 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c36d58-aa3e-4826-b667-9d2e68364a03 service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:23.011228 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c36d58-aa3e-4826-b667-9d2e68364a03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:23.031851 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 922/1863] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:23.039729 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c1591-91b2-4ad3-9538-e23c23576e6a None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:23.057798 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c1591-91b2-4ad3-9538-e23c23576e6a None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:23.066477 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3259e2d3-a5b4-4ef7-9a08-53d2fb1d3912 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:23.069119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3259e2d3-a5b4-4ef7-9a08-53d2fb1d3912 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:23.069596 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3259e2d3-a5b4-4ef7-9a08-53d2fb1d3912 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:23.070308 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3259e2d3-a5b4-4ef7-9a08-53d2fb1d3912 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:23.086517 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3259e2d3-a5b4-4ef7-9a08-53d2fb1d3912 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:23.090766 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3259e2d3-a5b4-4ef7-9a08-53d2fb1d3912 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:23.102669 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c36d58-aa3e-4826-b667-9d2e68364a03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:23.104225 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c36d58-aa3e-4826-b667-9d2e68364a03 service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:23.109282 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 923/1864] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:45:23.121745 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 942/1865] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:45:23.553273 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:23.922470 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3259e2d3-a5b4-4ef7-9a08-53d2fb1d3912 service neutron] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:23.926438 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-3259e2d3-a5b4-4ef7-9a08-53d2fb1d3912 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:23.946594 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:24.011198 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3259e2d3-a5b4-4ef7-9a08-53d2fb1d3912 service neutron] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:24.029069 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d651fa41-cb6b-486b-8d39-f60eca171675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:24.030178 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651fa41-cb6b-486b-8d39-f60eca171675 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:24.030753 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651fa41-cb6b-486b-8d39-f60eca171675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:24.031392 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651fa41-cb6b-486b-8d39-f60eca171675 admin admin] PATH_INFO: `/v3/projects/d449049a2c79446e90bcfe601c7bac5b` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:24.033710 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d651fa41-cb6b-486b-8d39-f60eca171675 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d449049a2c79446e90bcfe601c7bac5b)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:24.046870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d651fa41-cb6b-486b-8d39-f60eca171675 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:24.266221 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-d651fa41-cb6b-486b-8d39-f60eca171675 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd449049a2c79446e90bcfe601c7bac5b', 'request_id': 'req-d651fa41-cb6b-486b-8d39-f60eca171675'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:24.281614 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-d651fa41-cb6b-486b-8d39-f60eca171675 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd449049a2c79446e90bcfe601c7bac5b', 'request_id': 'req-d651fa41-cb6b-486b-8d39-f60eca171675'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:24.326380 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c36d58-aa3e-4826-b667-9d2e68364a03 service neutron] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:24.328161 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-59c36d58-aa3e-4826-b667-9d2e68364a03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:24.348048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c36d58-aa3e-4826-b667-9d2e68364a03 service neutron] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:24.364708 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-d651fa41-cb6b-486b-8d39-f60eca171675 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd449049a2c79446e90bcfe601c7bac5b', 'request_id': 'req-d651fa41-cb6b-486b-8d39-f60eca171675'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:24.383275 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-d651fa41-cb6b-486b-8d39-f60eca171675 admin admin] The token cache is being invalidate because project d449049a2c79446e90bcfe601c7bac5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:45:24.383975 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-d651fa41-cb6b-486b-8d39-f60eca171675 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:24.384823 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-d651fa41-cb6b-486b-8d39-f60eca171675 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:24.395139 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 924/1866] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:45:23 2022] DELETE /identity/v3/projects/d449049a2c79446e90bcfe601c7bac5b => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:45:24.399749 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d651fa41-cb6b-486b-8d39-f60eca171675 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:24.407307 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-d651fa41-cb6b-486b-8d39-f60eca171675 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:24.462109 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8592f34f-8ec9-4419-be75-5e814a7bef4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:24.463003 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592f34f-8ec9-4419-be75-5e814a7bef4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:24.463358 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592f34f-8ec9-4419-be75-5e814a7bef4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:24.463703 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592f34f-8ec9-4419-be75-5e814a7bef4a admin admin] PATH_INFO: `/v3/projects/6cf816bd33084fb5b3c9402edcb65788` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:24.465198 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8592f34f-8ec9-4419-be75-5e814a7bef4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cf816bd33084fb5b3c9402edcb65788)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:24.468009 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8592f34f-8ec9-4419-be75-5e814a7bef4a admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:24.516354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d651fa41-cb6b-486b-8d39-f60eca171675 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:24.521972 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d02146c-f512-4388-bf16-4528c875bd00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:24.522602 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02146c-f512-4388-bf16-4528c875bd00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:24.522887 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02146c-f512-4388-bf16-4528c875bd00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:24.523168 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02146c-f512-4388-bf16-4528c875bd00 admin admin] PATH_INFO: `/v3/users/4b9a15110190443c8950eb6861196205` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:24.524309 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d02146c-f512-4388-bf16-4528c875bd00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b9a15110190443c8950eb6861196205)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:24.526325 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d02146c-f512-4388-bf16-4528c875bd00 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:24.763280 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-8592f34f-8ec9-4419-be75-5e814a7bef4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cf816bd33084fb5b3c9402edcb65788', 'request_id': 'req-8592f34f-8ec9-4419-be75-5e814a7bef4a'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:24.771538 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-8592f34f-8ec9-4419-be75-5e814a7bef4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cf816bd33084fb5b3c9402edcb65788', 'request_id': 'req-8592f34f-8ec9-4419-be75-5e814a7bef4a'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:24.994402 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-8592f34f-8ec9-4419-be75-5e814a7bef4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cf816bd33084fb5b3c9402edcb65788', 'request_id': 'req-8592f34f-8ec9-4419-be75-5e814a7bef4a'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:24.995804 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-8592f34f-8ec9-4419-be75-5e814a7bef4a admin admin] The token cache is being invalidate because project 6cf816bd33084fb5b3c9402edcb65788 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:45:24.996107 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-8592f34f-8ec9-4419-be75-5e814a7bef4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:24.996890 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-8592f34f-8ec9-4419-be75-5e814a7bef4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:24.998824 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 943/1867] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:45:24 2022] DELETE /identity/v3/projects/6cf816bd33084fb5b3c9402edcb65788 => generated 0 bytes in 673 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:24.999652 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8592f34f-8ec9-4419-be75-5e814a7bef4a admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:25.001345 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-5d02146c-f512-4388-bf16-4528c875bd00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b9a15110190443c8950eb6861196205'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:25.003941 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-8592f34f-8ec9-4419-be75-5e814a7bef4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:25.015758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-5d02146c-f512-4388-bf16-4528c875bd00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b9a15110190443c8950eb6861196205'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:25.016537 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-5d02146c-f512-4388-bf16-4528c875bd00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b9a15110190443c8950eb6861196205'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:25.082139 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8592f34f-8ec9-4419-be75-5e814a7bef4a admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:25.089455 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d5df6-8e2b-404a-a6aa-cb46f0ccca13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:25.090294 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d5df6-8e2b-404a-a6aa-cb46f0ccca13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:25.090584 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d5df6-8e2b-404a-a6aa-cb46f0ccca13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:25.091291 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d5df6-8e2b-404a-a6aa-cb46f0ccca13 admin admin] PATH_INFO: `/v3/users/4a37e78c50614652adfc7330de6daa5f` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:25.096645 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d5df6-8e2b-404a-a6aa-cb46f0ccca13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a37e78c50614652adfc7330de6daa5f)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:25.096645 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d5df6-8e2b-404a-a6aa-cb46f0ccca13 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:25.149757 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-5d02146c-f512-4388-bf16-4528c875bd00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b9a15110190443c8950eb6861196205'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:25.194332 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-5d02146c-f512-4388-bf16-4528c875bd00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b9a15110190443c8950eb6861196205'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:25.224567 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 925/1868] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:45:24 2022] DELETE /identity/v3/users/4b9a15110190443c8950eb6861196205 => generated 0 bytes in 829 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:25.226041 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522846f6-0726-4c70-927a-c1fb73c2d6de None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:25.226498 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522846f6-0726-4c70-927a-c1fb73c2d6de None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:25.226925 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522846f6-0726-4c70-927a-c1fb73c2d6de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:25.260128 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-522846f6-0726-4c70-927a-c1fb73c2d6de None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:25.260128 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-522846f6-0726-4c70-927a-c1fb73c2d6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:25.345855 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 926/1869] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:25.349219 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91edd8c1-db31-43ed-a3f5-69a193e31c95 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:25.349219 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91edd8c1-db31-43ed-a3f5-69a193e31c95 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:25.349219 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91edd8c1-db31-43ed-a3f5-69a193e31c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:25.354901 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:25.373869 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-91edd8c1-db31-43ed-a3f5-69a193e31c95 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:25.375087 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-91edd8c1-db31-43ed-a3f5-69a193e31c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:25.389733 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 927/1870] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:25.389733 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e11b0-7555-46fb-a762-d00521a0cafd None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:25.389733 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e11b0-7555-46fb-a762-d00521a0cafd None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:25.389733 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e11b0-7555-46fb-a762-d00521a0cafd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:25.412173 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-027e11b0-7555-46fb-a762-d00521a0cafd None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:25.413891 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-027e11b0-7555-46fb-a762-d00521a0cafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:25.425462 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 928/1871] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:25.426446 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e11b0-7555-46fb-a762-d00521a0cafd None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:25.427876 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-027e11b0-7555-46fb-a762-d00521a0cafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:25.492978 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e11b0-7555-46fb-a762-d00521a0cafd None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:25.496217 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-054d5df6-8e2b-404a-a6aa-cb46f0ccca13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a37e78c50614652adfc7330de6daa5f'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:25.498178 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b414dcc1-eb3c-4ddd-a3ea-c9158b671509 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:25.498892 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414dcc1-eb3c-4ddd-a3ea-c9158b671509 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:25.499391 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414dcc1-eb3c-4ddd-a3ea-c9158b671509 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:25.499879 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414dcc1-eb3c-4ddd-a3ea-c9158b671509 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:25.515139 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b414dcc1-eb3c-4ddd-a3ea-c9158b671509 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:25.515139 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b414dcc1-eb3c-4ddd-a3ea-c9158b671509 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:25.524686 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 929/1872] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:45:25.525574 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b414dcc1-eb3c-4ddd-a3ea-c9158b671509 service neutron] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:25.536026 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b414dcc1-eb3c-4ddd-a3ea-c9158b671509 service neutron] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:25.541818 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963f5dd0-0fd0-46aa-9481-6a41a3234b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:25.542443 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f5dd0-0fd0-46aa-9481-6a41a3234b88 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:25.542722 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f5dd0-0fd0-46aa-9481-6a41a3234b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:25.542998 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f5dd0-0fd0-46aa-9481-6a41a3234b88 admin admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:25.543660 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963f5dd0-0fd0-46aa-9481-6a41a3234b88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:45:25.543967 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963f5dd0-0fd0-46aa-9481-6a41a3234b88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:25.545661 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963f5dd0-0fd0-46aa-9481-6a41a3234b88 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:25.560120 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 930/1873] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:45:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:45:25.565146 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963f5dd0-0fd0-46aa-9481-6a41a3234b88 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:25.576323 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963f5dd0-0fd0-46aa-9481-6a41a3234b88 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:25.582612 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db76d5ae-b3ed-4fad-9d08-544cddde7dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:25.583238 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76d5ae-b3ed-4fad-9d08-544cddde7dd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:25.583876 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76d5ae-b3ed-4fad-9d08-544cddde7dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:25.584379 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76d5ae-b3ed-4fad-9d08-544cddde7dd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:25.585603 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db76d5ae-b3ed-4fad-9d08-544cddde7dd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:25.589577 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db76d5ae-b3ed-4fad-9d08-544cddde7dd1 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:25.594819 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-054d5df6-8e2b-404a-a6aa-cb46f0ccca13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a37e78c50614652adfc7330de6daa5f'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:25.606866 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-054d5df6-8e2b-404a-a6aa-cb46f0ccca13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a37e78c50614652adfc7330de6daa5f'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:25.623021 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-054d5df6-8e2b-404a-a6aa-cb46f0ccca13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a37e78c50614652adfc7330de6daa5f'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:25.623970 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-054d5df6-8e2b-404a-a6aa-cb46f0ccca13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a37e78c50614652adfc7330de6daa5f'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:25.630491 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:25.637875 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 944/1874] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:45:24 2022] DELETE /identity/v3/users/4a37e78c50614652adfc7330de6daa5f => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:25.704483 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 931/1875] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:45:25 2022] POST /identity/v3/projects => generated 375 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:25.710643 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d5df6-8e2b-404a-a6aa-cb46f0ccca13 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:25.712986 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-054d5df6-8e2b-404a-a6aa-cb46f0ccca13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:25.826102 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d5df6-8e2b-404a-a6aa-cb46f0ccca13 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:25.912515 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6aa1d1-3dff-418f-9cfb-7d4201b23b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:25.913432 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6aa1d1-3dff-418f-9cfb-7d4201b23b52 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:25.914015 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6aa1d1-3dff-418f-9cfb-7d4201b23b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:25.914624 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6aa1d1-3dff-418f-9cfb-7d4201b23b52 admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:25.915421 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6aa1d1-3dff-418f-9cfb-7d4201b23b52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:25.918012 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6aa1d1-3dff-418f-9cfb-7d4201b23b52 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:26.050770 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 945/1876] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:45:25 2022] POST /identity/v3/users => generated 288 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:45:26.051589 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db76d5ae-b3ed-4fad-9d08-544cddde7dd1 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:26.063854 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db76d5ae-b3ed-4fad-9d08-544cddde7dd1 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:26.069940 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b05f6-b390-40da-bb89-a2265c475187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:26.070585 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b05f6-b390-40da-bb89-a2265c475187 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:26.070873 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b05f6-b390-40da-bb89-a2265c475187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:26.073685 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b05f6-b390-40da-bb89-a2265c475187 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:26.073685 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483b05f6-b390-40da-bb89-a2265c475187 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:26.073685 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483b05f6-b390-40da-bb89-a2265c475187 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:26.088279 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 932/1877] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:26.090114 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6aa1d1-3dff-418f-9cfb-7d4201b23b52 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:26.103048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6aa1d1-3dff-418f-9cfb-7d4201b23b52 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:26.107869 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f11f6-ad95-4ffa-9848-d81a825be773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:26.108618 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f11f6-ad95-4ffa-9848-d81a825be773 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:26.108798 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f11f6-ad95-4ffa-9848-d81a825be773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:26.109139 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f11f6-ad95-4ffa-9848-d81a825be773 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:26.109797 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2f11f6-ad95-4ffa-9848-d81a825be773 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:26.111630 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2f11f6-ad95-4ffa-9848-d81a825be773 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:26.123176 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 946/1878] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:26.127858 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b05f6-b390-40da-bb89-a2265c475187 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:26.142731 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b05f6-b390-40da-bb89-a2265c475187 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:26.149069 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf1ac95-60b6-424d-8b51-d154dc833162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:26.150023 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf1ac95-60b6-424d-8b51-d154dc833162 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:26.150516 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf1ac95-60b6-424d-8b51-d154dc833162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:26.150874 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf1ac95-60b6-424d-8b51-d154dc833162 admin admin] PATH_INFO: `/v3/projects/6e8667d341434925adf4096426e5b55b/users/5dc68e2ba7d44470b97b7a80a1808af5/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:26.169581 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf1ac95-60b6-424d-8b51-d154dc833162 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e8667d341434925adf4096426e5b55b, user_id=5dc68e2ba7d44470b97b7a80a1808af5, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:26.173529 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f11f6-ad95-4ffa-9848-d81a825be773 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:26.173759 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf1ac95-60b6-424d-8b51-d154dc833162 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:26.177609 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-fa2f11f6-ad95-4ffa-9848-d81a825be773 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:26.204286 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f11f6-ad95-4ffa-9848-d81a825be773 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:26.210811 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a789d74f-d408-4724-8be4-7ef48c55b59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:26.211563 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789d74f-d408-4724-8be4-7ef48c55b59a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:26.215034 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789d74f-d408-4724-8be4-7ef48c55b59a admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:26.215454 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789d74f-d408-4724-8be4-7ef48c55b59a admin admin] PATH_INFO: `/v3/projects/3f94937573cf4ee3907e3331c171df84` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:26.218581 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a789d74f-d408-4724-8be4-7ef48c55b59a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f94937573cf4ee3907e3331c171df84)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:26.222838 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a789d74f-d408-4724-8be4-7ef48c55b59a admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:26.311340 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 933/1879] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:26 2022] PUT /identity/v3/projects/6e8667d341434925adf4096426e5b55b/users/5dc68e2ba7d44470b97b7a80a1808af5/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:26.313474 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf1ac95-60b6-424d-8b51-d154dc833162 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:26.319355 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-eaf1ac95-60b6-424d-8b51-d154dc833162 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:26.418559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-a789d74f-d408-4724-8be4-7ef48c55b59a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f94937573cf4ee3907e3331c171df84', 'request_id': 'req-a789d74f-d408-4724-8be4-7ef48c55b59a'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:26.431575 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:26.436220 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-a789d74f-d408-4724-8be4-7ef48c55b59a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f94937573cf4ee3907e3331c171df84', 'request_id': 'req-a789d74f-d408-4724-8be4-7ef48c55b59a'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:26.457966 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf1ac95-60b6-424d-8b51-d154dc833162 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:26.520664 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-a789d74f-d408-4724-8be4-7ef48c55b59a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f94937573cf4ee3907e3331c171df84', 'request_id': 'req-a789d74f-d408-4724-8be4-7ef48c55b59a'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:26.542091 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-a789d74f-d408-4724-8be4-7ef48c55b59a admin admin] The token cache is being invalidate because project 3f94937573cf4ee3907e3331c171df84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:45:26.542445 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-a789d74f-d408-4724-8be4-7ef48c55b59a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:26.543656 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-a789d74f-d408-4724-8be4-7ef48c55b59a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:26.547819 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 947/1880] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:45:26 2022] DELETE /identity/v3/projects/3f94937573cf4ee3907e3331c171df84 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:45:26.551631 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b5a97-a5bc-4415-8ec1-394f8405f80d None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:26.552519 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b5a97-a5bc-4415-8ec1-394f8405f80d None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:26.553438 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b5a97-a5bc-4415-8ec1-394f8405f80d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:26.555099 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894c1a43-f431-4e9c-a8a0-008c087c11d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:26.556498 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c1a43-f431-4e9c-a8a0-008c087c11d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:26.556814 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c1a43-f431-4e9c-a8a0-008c087c11d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:26.557767 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c1a43-f431-4e9c-a8a0-008c087c11d4 admin admin] PATH_INFO: `/v3/projects/7d39e67cce00462aa26d741f375472ce` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:26.559646 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894c1a43-f431-4e9c-a8a0-008c087c11d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d39e67cce00462aa26d741f375472ce)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:26.566534 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894c1a43-f431-4e9c-a8a0-008c087c11d4 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:26.624356 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-5a4b5a97-a5bc-4415-8ec1-394f8405f80d None None] MFA Rules not processed for user `5dc68e2ba7d44470b97b7a80a1808af5`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:26.626515 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-5a4b5a97-a5bc-4415-8ec1-394f8405f80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:26.628800 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:26.728976 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 948/1881] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:26.731154 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b5a97-a5bc-4415-8ec1-394f8405f80d None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:26.733768 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-5a4b5a97-a5bc-4415-8ec1-394f8405f80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:26.746225 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-894c1a43-f431-4e9c-a8a0-008c087c11d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d39e67cce00462aa26d741f375472ce', 'request_id': 'req-894c1a43-f431-4e9c-a8a0-008c087c11d4'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:26.754500 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-894c1a43-f431-4e9c-a8a0-008c087c11d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d39e67cce00462aa26d741f375472ce', 'request_id': 'req-894c1a43-f431-4e9c-a8a0-008c087c11d4'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:26.761620 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b5a97-a5bc-4415-8ec1-394f8405f80d None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:26.839872 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b89db-3a83-42cf-8dce-cee4ef516c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:26.840613 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b89db-3a83-42cf-8dce-cee4ef516c01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:26.840902 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b89db-3a83-42cf-8dce-cee4ef516c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:26.841174 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b89db-3a83-42cf-8dce-cee4ef516c01 admin admin] PATH_INFO: `/v3/users/d2a559bf8f604af89a26c760799939a0` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:26.842368 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554b89db-3a83-42cf-8dce-cee4ef516c01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2a559bf8f604af89a26c760799939a0)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:26.844577 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554b89db-3a83-42cf-8dce-cee4ef516c01 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:26.875767 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-894c1a43-f431-4e9c-a8a0-008c087c11d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d39e67cce00462aa26d741f375472ce', 'request_id': 'req-894c1a43-f431-4e9c-a8a0-008c087c11d4'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:26.889672 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-894c1a43-f431-4e9c-a8a0-008c087c11d4 admin admin] The token cache is being invalidate because project 7d39e67cce00462aa26d741f375472ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:45:26.889963 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-894c1a43-f431-4e9c-a8a0-008c087c11d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:26.890666 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-894c1a43-f431-4e9c-a8a0-008c087c11d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:26.892224 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 934/1882] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:45:26 2022] DELETE /identity/v3/projects/7d39e67cce00462aa26d741f375472ce => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:26.893629 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894c1a43-f431-4e9c-a8a0-008c087c11d4 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:26.895120 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-894c1a43-f431-4e9c-a8a0-008c087c11d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:26.908564 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894c1a43-f431-4e9c-a8a0-008c087c11d4 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:26.913773 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372586cd-4ce6-4a8c-aad0-ba26f1cda1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:26.914393 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372586cd-4ce6-4a8c-aad0-ba26f1cda1cc admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:26.914677 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372586cd-4ce6-4a8c-aad0-ba26f1cda1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:26.914965 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372586cd-4ce6-4a8c-aad0-ba26f1cda1cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:26.915602 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372586cd-4ce6-4a8c-aad0-ba26f1cda1cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:26.917836 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372586cd-4ce6-4a8c-aad0-ba26f1cda1cc admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:27.011137 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 935/1883] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:45:26 2022] POST /identity/v3/projects => generated 375 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:27.014102 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa0f1e-7503-4366-bc9b-79e275a438ff None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:27.014934 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa0f1e-7503-4366-bc9b-79e275a438ff None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:27.015933 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa0f1e-7503-4366-bc9b-79e275a438ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:27.042906 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-55fa0f1e-7503-4366-bc9b-79e275a438ff None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:27.044246 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-55fa0f1e-7503-4366-bc9b-79e275a438ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:27.123238 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-554b89db-3a83-42cf-8dce-cee4ef516c01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2a559bf8f604af89a26c760799939a0'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:27.124052 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 936/1884] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:27.125184 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa0f1e-7503-4366-bc9b-79e275a438ff None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:27.139359 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa0f1e-7503-4366-bc9b-79e275a438ff None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:27.146998 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219a59cd-8938-498d-a8c3-065c53830267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:27.147701 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a59cd-8938-498d-a8c3-065c53830267 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:27.148088 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a59cd-8938-498d-a8c3-065c53830267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:27.148454 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a59cd-8938-498d-a8c3-065c53830267 admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:27.149178 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219a59cd-8938-498d-a8c3-065c53830267 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:27.156279 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219a59cd-8938-498d-a8c3-065c53830267 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:27.206720 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-554b89db-3a83-42cf-8dce-cee4ef516c01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2a559bf8f604af89a26c760799939a0'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:27.220825 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-554b89db-3a83-42cf-8dce-cee4ef516c01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2a559bf8f604af89a26c760799939a0'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:27.239967 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-554b89db-3a83-42cf-8dce-cee4ef516c01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2a559bf8f604af89a26c760799939a0'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:27.240873 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-554b89db-3a83-42cf-8dce-cee4ef516c01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2a559bf8f604af89a26c760799939a0'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:27.245267 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 937/1885] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:45:27 2022] POST /identity/v3/users => generated 288 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:27.248586 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9155a78-9b6a-41e7-b856-85478bc8ec84 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:27.248759 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 949/1886] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:45:26 2022] DELETE /identity/v3/users/d2a559bf8f604af89a26c760799939a0 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:27.249475 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9155a78-9b6a-41e7-b856-85478bc8ec84 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:27.249974 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9155a78-9b6a-41e7-b856-85478bc8ec84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:27.255928 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b89db-3a83-42cf-8dce-cee4ef516c01 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:27.255928 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-554b89db-3a83-42cf-8dce-cee4ef516c01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:27.284681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-c9155a78-9b6a-41e7-b856-85478bc8ec84 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:27.287044 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-c9155a78-9b6a-41e7-b856-85478bc8ec84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:27.370767 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b89db-3a83-42cf-8dce-cee4ef516c01 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:27.380449 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9614af5-8d8e-4a82-9082-60c37f84b82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:27.382385 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9614af5-8d8e-4a82-9082-60c37f84b82a admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:27.382770 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9614af5-8d8e-4a82-9082-60c37f84b82a admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:27.383215 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9614af5-8d8e-4a82-9082-60c37f84b82a admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:27.384527 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9614af5-8d8e-4a82-9082-60c37f84b82a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:27.386992 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9614af5-8d8e-4a82-9082-60c37f84b82a admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:27.399915 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 938/1887] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:27.400976 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 950/1888] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:27 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:27.405194 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9155a78-9b6a-41e7-b856-85478bc8ec84 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:27.405580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9614af5-8d8e-4a82-9082-60c37f84b82a admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:27.418624 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9155a78-9b6a-41e7-b856-85478bc8ec84 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:27.418980 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9614af5-8d8e-4a82-9082-60c37f84b82a admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:27.425677 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f347b13-4104-4dbb-a3bd-70fecbb529e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:27.426870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f347b13-4104-4dbb-a3bd-70fecbb529e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:27.427169 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f347b13-4104-4dbb-a3bd-70fecbb529e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:27.427348 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c2e662-78c2-4f5f-80ed-707240cad0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:27.427525 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f347b13-4104-4dbb-a3bd-70fecbb529e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:27.428434 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f347b13-4104-4dbb-a3bd-70fecbb529e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:45:27.428606 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c2e662-78c2-4f5f-80ed-707240cad0a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:27.428691 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f347b13-4104-4dbb-a3bd-70fecbb529e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:27.428763 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c2e662-78c2-4f5f-80ed-707240cad0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:27.429138 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c2e662-78c2-4f5f-80ed-707240cad0a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:27.430325 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f347b13-4104-4dbb-a3bd-70fecbb529e3 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:27.431251 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c2e662-78c2-4f5f-80ed-707240cad0a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:27.433986 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c2e662-78c2-4f5f-80ed-707240cad0a3 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:27.442279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:27.444448 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 939/1889] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:45:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:45:27.450149 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 951/1890] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:27 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:45:27.455701 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f347b13-4104-4dbb-a3bd-70fecbb529e3 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:27.457496 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c2e662-78c2-4f5f-80ed-707240cad0a3 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:27.468439 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c2e662-78c2-4f5f-80ed-707240cad0a3 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:27.473126 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f347b13-4104-4dbb-a3bd-70fecbb529e3 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:27.474363 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a2861d-2359-4fc8-8585-00b318464550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:27.474986 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2861d-2359-4fc8-8585-00b318464550 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:27.475380 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2861d-2359-4fc8-8585-00b318464550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:27.476717 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2861d-2359-4fc8-8585-00b318464550 admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:27.476717 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a2861d-2359-4fc8-8585-00b318464550 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:27.478694 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a2861d-2359-4fc8-8585-00b318464550 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:27.480716 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4385870-690e-4598-b6f5-0e6a955db5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:27.481712 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4385870-690e-4598-b6f5-0e6a955db5d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:27.482164 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4385870-690e-4598-b6f5-0e6a955db5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:27.482650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4385870-690e-4598-b6f5-0e6a955db5d9 admin admin] PATH_INFO: `/v3/projects/0074b22dffe2425f90dbd82c0ed91b67/users/74c44a550c54451aae6bc751b57c7b4a/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:27.503260 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4385870-690e-4598-b6f5-0e6a955db5d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0074b22dffe2425f90dbd82c0ed91b67, user_id=74c44a550c54451aae6bc751b57c7b4a, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:27.508176 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4385870-690e-4598-b6f5-0e6a955db5d9 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:27.574503 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 952/1891] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:45:27 2022] POST /identity/v3/projects => generated 381 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:45:27.578966 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a2861d-2359-4fc8-8585-00b318464550 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:27.665435 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a2861d-2359-4fc8-8585-00b318464550 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:27.673484 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432624cf-5635-42d4-bfd9-dea0c50f6e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:27.674586 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432624cf-5635-42d4-bfd9-dea0c50f6e3e admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:27.676099 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432624cf-5635-42d4-bfd9-dea0c50f6e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:27.676723 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432624cf-5635-42d4-bfd9-dea0c50f6e3e admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:27.678308 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432624cf-5635-42d4-bfd9-dea0c50f6e3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:27.682272 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432624cf-5635-42d4-bfd9-dea0c50f6e3e admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:27.686128 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 940/1892] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:27 2022] PUT /identity/v3/projects/0074b22dffe2425f90dbd82c0ed91b67/users/74c44a550c54451aae6bc751b57c7b4a/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:45:27.703596 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e988e-231d-4c33-8f41-c52aeb039d6f None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:27.704159 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e988e-231d-4c33-8f41-c52aeb039d6f None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:27.704499 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e988e-231d-4c33-8f41-c52aeb039d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:27.753938 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-9f1e988e-231d-4c33-8f41-c52aeb039d6f None None] MFA Rules not processed for user `74c44a550c54451aae6bc751b57c7b4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:27.757846 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:27.759027 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-9f1e988e-231d-4c33-8f41-c52aeb039d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:27.778734 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 953/1893] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:45:27 2022] POST /identity/v3/users => generated 291 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:27.783071 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432624cf-5635-42d4-bfd9-dea0c50f6e3e admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:27.823716 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432624cf-5635-42d4-bfd9-dea0c50f6e3e admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:27.920503 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d364c954-829f-4272-b854-94406b8dbe31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:27.921131 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364c954-829f-4272-b854-94406b8dbe31 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:27.921691 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 941/1894] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:45:27.921846 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364c954-829f-4272-b854-94406b8dbe31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:27.921846 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364c954-829f-4272-b854-94406b8dbe31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:27.925025 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d364c954-829f-4272-b854-94406b8dbe31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:27.927674 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d364c954-829f-4272-b854-94406b8dbe31 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:27.931603 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e988e-231d-4c33-8f41-c52aeb039d6f None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:27.944302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e988e-231d-4c33-8f41-c52aeb039d6f None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:27.945084 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 954/1895] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:27 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:45:27.949151 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d364c954-829f-4272-b854-94406b8dbe31 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:27.954738 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af73e76c-6988-42cb-b70a-4835452b1a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:27.963886 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73e76c-6988-42cb-b70a-4835452b1a3f admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:27.963886 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73e76c-6988-42cb-b70a-4835452b1a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:27.963886 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73e76c-6988-42cb-b70a-4835452b1a3f admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:27.963886 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af73e76c-6988-42cb-b70a-4835452b1a3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:27.963886 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af73e76c-6988-42cb-b70a-4835452b1a3f admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:27.967460 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d364c954-829f-4272-b854-94406b8dbe31 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:27.971935 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82788fb8-292a-4c80-9076-10683954b65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:27.972718 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82788fb8-292a-4c80-9076-10683954b65d admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:27.973130 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82788fb8-292a-4c80-9076-10683954b65d admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:27.973487 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82788fb8-292a-4c80-9076-10683954b65d admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:27.974474 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82788fb8-292a-4c80-9076-10683954b65d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:27.976331 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82788fb8-292a-4c80-9076-10683954b65d admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:27.990610 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 955/1896] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:27 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:45:27.995442 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82788fb8-292a-4c80-9076-10683954b65d admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:28.007905 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82788fb8-292a-4c80-9076-10683954b65d admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:28.028288 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8f269e-09c1-4225-bf62-916b4a8acffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:28.029971 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f269e-09c1-4225-bf62-916b4a8acffe admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:28.030449 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f269e-09c1-4225-bf62-916b4a8acffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:28.030967 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f269e-09c1-4225-bf62-916b4a8acffe admin admin] PATH_INFO: `/v3/projects/b09f9c6ff4e24b16be8d00931698b465/users/6ead5bd6b21b46ab9b839fdef9be99ef/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:28.063918 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8f269e-09c1-4225-bf62-916b4a8acffe admin admin] RBAC: Authorizing `identity:create_grant(project_id=b09f9c6ff4e24b16be8d00931698b465, user_id=6ead5bd6b21b46ab9b839fdef9be99ef, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:28.067335 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8f269e-09c1-4225-bf62-916b4a8acffe admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:28.082576 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 942/1897] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:45:27 2022] POST /identity/v3/projects => generated 375 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:28.087019 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af73e76c-6988-42cb-b70a-4835452b1a3f admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:28.200221 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 956/1898] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:27 2022] PUT /identity/v3/projects/b09f9c6ff4e24b16be8d00931698b465/users/6ead5bd6b21b46ab9b839fdef9be99ef/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:45:28.206053 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21871d9b-516a-454e-bb4a-58c41d2b5608 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:28.206943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21871d9b-516a-454e-bb4a-58c41d2b5608 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:28.206943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21871d9b-516a-454e-bb4a-58c41d2b5608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:28.234691 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af73e76c-6988-42cb-b70a-4835452b1a3f admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:28.276062 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-21871d9b-516a-454e-bb4a-58c41d2b5608 None None] MFA Rules not processed for user `6ead5bd6b21b46ab9b839fdef9be99ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:28.278065 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-21871d9b-516a-454e-bb4a-58c41d2b5608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:28.328853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e3db6-c82d-4bc3-9870-a46dbbb119db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:28.329565 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e3db6-c82d-4bc3-9870-a46dbbb119db admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:28.330057 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e3db6-c82d-4bc3-9870-a46dbbb119db admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:28.330588 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e3db6-c82d-4bc3-9870-a46dbbb119db admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:28.331859 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785e3db6-c82d-4bc3-9870-a46dbbb119db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:28.334523 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785e3db6-c82d-4bc3-9870-a46dbbb119db admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:28.416823 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 957/1899] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:28 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:28.436699 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21871d9b-516a-454e-bb4a-58c41d2b5608 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:28.438006 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21871d9b-516a-454e-bb4a-58c41d2b5608 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:28.443395 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9bf52f-ed41-4d16-a080-94b18df71f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:28.444093 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9bf52f-ed41-4d16-a080-94b18df71f31 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:28.444322 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9bf52f-ed41-4d16-a080-94b18df71f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:28.444671 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9bf52f-ed41-4d16-a080-94b18df71f31 admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:28.445753 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9bf52f-ed41-4d16-a080-94b18df71f31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:28.447739 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9bf52f-ed41-4d16-a080-94b18df71f31 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:28.669130 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 943/1900] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:45:28 2022] POST /identity/v3/users => generated 294 bytes in 582 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:28.711457 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e3db6-c82d-4bc3-9870-a46dbbb119db admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:28.722969 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e3db6-c82d-4bc3-9870-a46dbbb119db admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:28.727987 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a8163-9561-4eaa-8e5f-3cddb3d817ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:28.728733 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a8163-9561-4eaa-8e5f-3cddb3d817ae admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:28.729017 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a8163-9561-4eaa-8e5f-3cddb3d817ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:28.729290 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a8163-9561-4eaa-8e5f-3cddb3d817ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:28.729942 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190a8163-9561-4eaa-8e5f-3cddb3d817ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:28.731754 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190a8163-9561-4eaa-8e5f-3cddb3d817ae admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:28.744608 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 944/1901] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:45:28.751781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a8163-9561-4eaa-8e5f-3cddb3d817ae admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:28.755534 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 958/1902] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:45:28 2022] POST /identity/v3/projects => generated 381 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:45:28.759652 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9bf52f-ed41-4d16-a080-94b18df71f31 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:28.762603 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-be9bf52f-ed41-4d16-a080-94b18df71f31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:28.765355 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a8163-9561-4eaa-8e5f-3cddb3d817ae admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:28.844748 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9261faf1-8a1e-40f6-94c3-f3c72d283ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:28.845450 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261faf1-8a1e-40f6-94c3-f3c72d283ba8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:28.845738 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261faf1-8a1e-40f6-94c3-f3c72d283ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:28.846257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261faf1-8a1e-40f6-94c3-f3c72d283ba8 admin admin] PATH_INFO: `/v3/projects/8914bed0cb6e4431934ab70982b001c4/users/cf107eb0d6644b8da5e8877fb8bf9b9c/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:28.860628 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9bf52f-ed41-4d16-a080-94b18df71f31 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:28.861147 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9261faf1-8a1e-40f6-94c3-f3c72d283ba8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8914bed0cb6e4431934ab70982b001c4, user_id=cf107eb0d6644b8da5e8877fb8bf9b9c, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:28.863924 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9261faf1-8a1e-40f6-94c3-f3c72d283ba8 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:28.866931 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13bccb-51e2-4074-ac16-455e402ae4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:28.867594 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13bccb-51e2-4074-ac16-455e402ae4e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:28.867880 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13bccb-51e2-4074-ac16-455e402ae4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:28.868167 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13bccb-51e2-4074-ac16-455e402ae4e2 admin admin] PATH_INFO: `/v3/projects/5ddc63f6f46e455780ace9ab6a871913` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:28.869370 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b13bccb-51e2-4074-ac16-455e402ae4e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ddc63f6f46e455780ace9ab6a871913)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:28.871415 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b13bccb-51e2-4074-ac16-455e402ae4e2 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:29.030189 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 945/1903] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:28 2022] PUT /identity/v3/projects/8914bed0cb6e4431934ab70982b001c4/users/cf107eb0d6644b8da5e8877fb8bf9b9c/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:45:29.030430 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9261faf1-8a1e-40f6-94c3-f3c72d283ba8 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:29.085956 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-9b13bccb-51e2-4074-ac16-455e402ae4e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ddc63f6f46e455780ace9ab6a871913', 'request_id': 'req-9b13bccb-51e2-4074-ac16-455e402ae4e2'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:29.108278 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-9b13bccb-51e2-4074-ac16-455e402ae4e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ddc63f6f46e455780ace9ab6a871913', 'request_id': 'req-9b13bccb-51e2-4074-ac16-455e402ae4e2'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:29.163350 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9261faf1-8a1e-40f6-94c3-f3c72d283ba8 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:29.195496 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-9b13bccb-51e2-4074-ac16-455e402ae4e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ddc63f6f46e455780ace9ab6a871913', 'request_id': 'req-9b13bccb-51e2-4074-ac16-455e402ae4e2'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:29.199789 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:29.222146 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-9b13bccb-51e2-4074-ac16-455e402ae4e2 admin admin] The token cache is being invalidate because project 5ddc63f6f46e455780ace9ab6a871913 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:45:29.222834 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-9b13bccb-51e2-4074-ac16-455e402ae4e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:29.223736 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-9b13bccb-51e2-4074-ac16-455e402ae4e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:29.231130 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 959/1904] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:45:28 2022] DELETE /identity/v3/projects/5ddc63f6f46e455780ace9ab6a871913 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:29.233477 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13bccb-51e2-4074-ac16-455e402ae4e2 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:29.235358 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-9b13bccb-51e2-4074-ac16-455e402ae4e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:29.299056 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c38d4-87fb-471d-b8c5-49508fc927fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:29.299056 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c38d4-87fb-471d-b8c5-49508fc927fa admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:29.299056 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c38d4-87fb-471d-b8c5-49508fc927fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:29.299056 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c38d4-87fb-471d-b8c5-49508fc927fa admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:29.300041 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c38d4-87fb-471d-b8c5-49508fc927fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:29.300041 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c38d4-87fb-471d-b8c5-49508fc927fa admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:29.363601 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13bccb-51e2-4074-ac16-455e402ae4e2 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:29.371960 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46978417-507e-4172-8989-02f54efd5fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:29.372684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46978417-507e-4172-8989-02f54efd5fc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:29.372970 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46978417-507e-4172-8989-02f54efd5fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:29.373240 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46978417-507e-4172-8989-02f54efd5fc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:29.373936 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46978417-507e-4172-8989-02f54efd5fc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:29.375780 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46978417-507e-4172-8989-02f54efd5fc8 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:29.388876 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 960/1905] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:29.390294 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46978417-507e-4172-8989-02f54efd5fc8 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:29.392703 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-46978417-507e-4172-8989-02f54efd5fc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:29.408891 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46978417-507e-4172-8989-02f54efd5fc8 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:29.414886 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bba769-b72f-4872-97d5-8efb515491cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:29.415967 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bba769-b72f-4872-97d5-8efb515491cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:29.416277 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bba769-b72f-4872-97d5-8efb515491cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:29.416629 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bba769-b72f-4872-97d5-8efb515491cf admin admin] PATH_INFO: `/v3/users/c378ee0a0139443da7462ed19adb46f4` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:29.418167 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bba769-b72f-4872-97d5-8efb515491cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=c378ee0a0139443da7462ed19adb46f4)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:29.421894 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bba769-b72f-4872-97d5-8efb515491cf admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:29.429254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 946/1906] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:45:29 2022] POST /identity/v3/users => generated 297 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:29.430761 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c38d4-87fb-471d-b8c5-49508fc927fa admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:29.450957 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c38d4-87fb-471d-b8c5-49508fc927fa admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:29.456521 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5967fa-07e6-4d0b-8e48-54b119a18473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:29.457439 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5967fa-07e6-4d0b-8e48-54b119a18473 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:29.457940 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5967fa-07e6-4d0b-8e48-54b119a18473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:29.458388 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5967fa-07e6-4d0b-8e48-54b119a18473 admin admin] PATH_INFO: `/v3/projects/8914bed0cb6e4431934ab70982b001c4/users/cf107eb0d6644b8da5e8877fb8bf9b9c/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:29.462013 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5967fa-07e6-4d0b-8e48-54b119a18473 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8914bed0cb6e4431934ab70982b001c4, user_id=cf107eb0d6644b8da5e8877fb8bf9b9c, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:29.464905 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5967fa-07e6-4d0b-8e48-54b119a18473 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:29.477319 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 947/1907] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:29 2022] PUT /identity/v3/projects/8914bed0cb6e4431934ab70982b001c4/users/cf107eb0d6644b8da5e8877fb8bf9b9c/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:29.478652 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5967fa-07e6-4d0b-8e48-54b119a18473 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:29.480828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-0d5967fa-07e6-4d0b-8e48-54b119a18473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:29.572524 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5967fa-07e6-4d0b-8e48-54b119a18473 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:29.578363 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968ff60a-b97d-403c-80e0-dd3d2265619c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:29.578997 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ff60a-b97d-403c-80e0-dd3d2265619c admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:29.579280 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ff60a-b97d-403c-80e0-dd3d2265619c admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:29.579566 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ff60a-b97d-403c-80e0-dd3d2265619c admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:29.580348 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968ff60a-b97d-403c-80e0-dd3d2265619c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:29.582275 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968ff60a-b97d-403c-80e0-dd3d2265619c admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:29.599393 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 948/1908] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:29.600824 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13daed7-b9e7-4bbc-8a54-1fda4da4550f None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:29.601158 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13daed7-b9e7-4bbc-8a54-1fda4da4550f None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:29.601432 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13daed7-b9e7-4bbc-8a54-1fda4da4550f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:29.660735 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-f13daed7-b9e7-4bbc-8a54-1fda4da4550f None None] MFA Rules not processed for user `cf107eb0d6644b8da5e8877fb8bf9b9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:29.670476 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-f13daed7-b9e7-4bbc-8a54-1fda4da4550f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:29.685168 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:29.714424 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-97bba769-b72f-4872-97d5-8efb515491cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c378ee0a0139443da7462ed19adb46f4'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:29.824160 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-97bba769-b72f-4872-97d5-8efb515491cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c378ee0a0139443da7462ed19adb46f4'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:29.835922 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-97bba769-b72f-4872-97d5-8efb515491cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c378ee0a0139443da7462ed19adb46f4'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:29.844898 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 949/1909] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:29 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:29.849842 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13daed7-b9e7-4bbc-8a54-1fda4da4550f None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:29.858046 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-97bba769-b72f-4872-97d5-8efb515491cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c378ee0a0139443da7462ed19adb46f4'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:29.859042 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-97bba769-b72f-4872-97d5-8efb515491cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c378ee0a0139443da7462ed19adb46f4'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:29.872918 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 961/1910] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:45:29 2022] DELETE /identity/v3/users/c378ee0a0139443da7462ed19adb46f4 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:29.874020 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13daed7-b9e7-4bbc-8a54-1fda4da4550f None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:29.875401 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bba769-b72f-4872-97d5-8efb515491cf admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:29.879509 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-97bba769-b72f-4872-97d5-8efb515491cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:29.994471 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bba769-b72f-4872-97d5-8efb515491cf admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:29.999809 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06137847-b44b-4f5b-b6b9-a23ce80e1b85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:30.000749 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06137847-b44b-4f5b-b6b9-a23ce80e1b85 service cinder] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:30.001186 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06137847-b44b-4f5b-b6b9-a23ce80e1b85 service cinder] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:30.001614 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06137847-b44b-4f5b-b6b9-a23ce80e1b85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:30.005421 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-06137847-b44b-4f5b-b6b9-a23ce80e1b85 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:30.007212 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5001b89-8d71-4eb7-bd47-c6dd12f66db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:30.007900 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5001b89-8d71-4eb7-bd47-c6dd12f66db2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:30.008246 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5001b89-8d71-4eb7-bd47-c6dd12f66db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:30.008639 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5001b89-8d71-4eb7-bd47-c6dd12f66db2 admin admin] PATH_INFO: `/v3/projects/8550878d05a349fd9ac882ed7848b606/users/961b21e3f0bc4121ae84b9ddcec2d57e/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:30.044506 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5001b89-8d71-4eb7-bd47-c6dd12f66db2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8550878d05a349fd9ac882ed7848b606, user_id=961b21e3f0bc4121ae84b9ddcec2d57e, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:30.044506 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5001b89-8d71-4eb7-bd47-c6dd12f66db2 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:30.121859 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06137847-b44b-4f5b-b6b9-a23ce80e1b85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:30.123618 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06137847-b44b-4f5b-b6b9-a23ce80e1b85 service cinder] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:30.134026 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 962/1911] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:45:30.198442 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 950/1912] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:29 2022] PUT /identity/v3/projects/8550878d05a349fd9ac882ed7848b606/users/961b21e3f0bc4121ae84b9ddcec2d57e/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:30.204945 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06137847-b44b-4f5b-b6b9-a23ce80e1b85 service cinder] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:30.207887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-06137847-b44b-4f5b-b6b9-a23ce80e1b85 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:30.252602 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06137847-b44b-4f5b-b6b9-a23ce80e1b85 service cinder] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:30.368697 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399eddca-30ae-43b4-b32c-2d7daa80d9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:30.372376 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399eddca-30ae-43b4-b32c-2d7daa80d9d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:30.372376 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399eddca-30ae-43b4-b32c-2d7daa80d9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:30.372376 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399eddca-30ae-43b4-b32c-2d7daa80d9d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:30.374274 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399eddca-30ae-43b4-b32c-2d7daa80d9d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:30.380451 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399eddca-30ae-43b4-b32c-2d7daa80d9d0 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:30.414910 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 963/1913] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:30 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:30.421084 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5001b89-8d71-4eb7-bd47-c6dd12f66db2 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:30.443857 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5001b89-8d71-4eb7-bd47-c6dd12f66db2 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:30.458917 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc4b8f7-e924-4924-b2fa-548a13e7af01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:30.459753 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4b8f7-e924-4924-b2fa-548a13e7af01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:30.460200 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4b8f7-e924-4924-b2fa-548a13e7af01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:30.460691 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4b8f7-e924-4924-b2fa-548a13e7af01 admin admin] PATH_INFO: `/v3/projects/8550878d05a349fd9ac882ed7848b606/users/961b21e3f0bc4121ae84b9ddcec2d57e/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:30.469076 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc4b8f7-e924-4924-b2fa-548a13e7af01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8550878d05a349fd9ac882ed7848b606, user_id=961b21e3f0bc4121ae84b9ddcec2d57e, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:30.472259 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc4b8f7-e924-4924-b2fa-548a13e7af01 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:30.515707 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 951/1914] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:30 2022] PUT /identity/v3/projects/8550878d05a349fd9ac882ed7848b606/users/961b21e3f0bc4121ae84b9ddcec2d57e/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:30.518232 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb5afe-4ea8-41a6-a3ac-89507c130c1c None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:30.518537 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb5afe-4ea8-41a6-a3ac-89507c130c1c None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:30.518962 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb5afe-4ea8-41a6-a3ac-89507c130c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:30.571618 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-62bb5afe-4ea8-41a6-a3ac-89507c130c1c None None] MFA Rules not processed for user `961b21e3f0bc4121ae84b9ddcec2d57e`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:30.573025 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-62bb5afe-4ea8-41a6-a3ac-89507c130c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:30.660329 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc4b8f7-e924-4924-b2fa-548a13e7af01 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:30.662392 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-abc4b8f7-e924-4924-b2fa-548a13e7af01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:30.670992 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 964/1915] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:30 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:45:30.691187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb5afe-4ea8-41a6-a3ac-89507c130c1c None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:30.697310 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-62bb5afe-4ea8-41a6-a3ac-89507c130c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:30.799370 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc4b8f7-e924-4924-b2fa-548a13e7af01 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:30.800847 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb5afe-4ea8-41a6-a3ac-89507c130c1c None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:30.805129 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c2907-6523-4867-8203-cf46cc591d49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:30.805773 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c2907-6523-4867-8203-cf46cc591d49 service cinder] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:30.806057 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c2907-6523-4867-8203-cf46cc591d49 service cinder] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:30.806335 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c2907-6523-4867-8203-cf46cc591d49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:30.809749 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03308899-3520-48d2-b3f9-5a73cebc0afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:30.810750 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03308899-3520-48d2-b3f9-5a73cebc0afd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:30.811138 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03308899-3520-48d2-b3f9-5a73cebc0afd admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:30.811505 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03308899-3520-48d2-b3f9-5a73cebc0afd admin admin] PATH_INFO: `/v3/projects/502817dabbac489997a3c7de0dcebc69` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:30.813063 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03308899-3520-48d2-b3f9-5a73cebc0afd admin admin] RBAC: Authorizing `identity:delete_project(project_id=502817dabbac489997a3c7de0dcebc69)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:30.815508 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03308899-3520-48d2-b3f9-5a73cebc0afd admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:30.817692 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71c2907-6523-4867-8203-cf46cc591d49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:30.819158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71c2907-6523-4867-8203-cf46cc591d49 service cinder] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:30.833521 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 965/1916] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:30 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:45:30.933092 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-03308899-3520-48d2-b3f9-5a73cebc0afd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '502817dabbac489997a3c7de0dcebc69', 'request_id': 'req-03308899-3520-48d2-b3f9-5a73cebc0afd'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:30.946751 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-03308899-3520-48d2-b3f9-5a73cebc0afd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '502817dabbac489997a3c7de0dcebc69', 'request_id': 'req-03308899-3520-48d2-b3f9-5a73cebc0afd'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:31.058939 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-03308899-3520-48d2-b3f9-5a73cebc0afd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '502817dabbac489997a3c7de0dcebc69', 'request_id': 'req-03308899-3520-48d2-b3f9-5a73cebc0afd'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:31.082982 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-03308899-3520-48d2-b3f9-5a73cebc0afd admin admin] The token cache is being invalidate because project 502817dabbac489997a3c7de0dcebc69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:45:31.083256 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-03308899-3520-48d2-b3f9-5a73cebc0afd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:31.084484 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-03308899-3520-48d2-b3f9-5a73cebc0afd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:31.087627 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 952/1917] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:45:30 2022] DELETE /identity/v3/projects/502817dabbac489997a3c7de0dcebc69 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:31.099634 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ad843-bce5-4800-9c86-a6adac9ed7ce None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:31.099925 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ad843-bce5-4800-9c86-a6adac9ed7ce None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:31.100435 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ad843-bce5-4800-9c86-a6adac9ed7ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:31.158416 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-5c7ad843-bce5-4800-9c86-a6adac9ed7ce None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:31.163768 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-5c7ad843-bce5-4800-9c86-a6adac9ed7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:31.280121 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 966/1918] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:31.289874 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f52d4-f7b6-4b09-99e0-a12b21faeed8 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:31.290162 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f52d4-f7b6-4b09-99e0-a12b21faeed8 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:31.290453 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f52d4-f7b6-4b09-99e0-a12b21faeed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:31.325610 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-b62f52d4-f7b6-4b09-99e0-a12b21faeed8 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:31.326871 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-b62f52d4-f7b6-4b09-99e0-a12b21faeed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:31.336981 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 953/1919] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:31.341692 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ad843-bce5-4800-9c86-a6adac9ed7ce None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:31.353699 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ad843-bce5-4800-9c86-a6adac9ed7ce None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:31.358989 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa510a-31ef-4f73-abde-3d4a9138e09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:31.359646 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa510a-31ef-4f73-abde-3d4a9138e09f admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:31.359946 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa510a-31ef-4f73-abde-3d4a9138e09f admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:31.364264 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa510a-31ef-4f73-abde-3d4a9138e09f admin admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:31.364264 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baa510a-31ef-4f73-abde-3d4a9138e09f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:45:31.364264 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baa510a-31ef-4f73-abde-3d4a9138e09f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:31.364264 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baa510a-31ef-4f73-abde-3d4a9138e09f admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:31.378160 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 967/1920] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:45:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:45:31.384197 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f52d4-f7b6-4b09-99e0-a12b21faeed8 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:31.395709 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f52d4-f7b6-4b09-99e0-a12b21faeed8 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:31.401338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd97061-80a5-4882-a153-a8e00df648da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:31.401985 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd97061-80a5-4882-a153-a8e00df648da admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:31.402273 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd97061-80a5-4882-a153-a8e00df648da admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:31.402543 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd97061-80a5-4882-a153-a8e00df648da admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:31.403215 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd97061-80a5-4882-a153-a8e00df648da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:31.405294 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd97061-80a5-4882-a153-a8e00df648da admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:31.423179 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:31.569786 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 954/1921] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:45:31 2022] POST /identity/v3/projects => generated 387 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:31.578574 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa510a-31ef-4f73-abde-3d4a9138e09f admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:31.598443 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa510a-31ef-4f73-abde-3d4a9138e09f admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:31.685958 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4158f7-3b0d-4b2b-97cf-07f48e9b6d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:31.686604 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4158f7-3b0d-4b2b-97cf-07f48e9b6d91 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:31.686885 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4158f7-3b0d-4b2b-97cf-07f48e9b6d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:31.687165 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4158f7-3b0d-4b2b-97cf-07f48e9b6d91 admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:31.687835 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4158f7-3b0d-4b2b-97cf-07f48e9b6d91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:31.690021 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4158f7-3b0d-4b2b-97cf-07f48e9b6d91 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:31.829488 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 968/1922] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:45:31 2022] POST /identity/v3/users => generated 294 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:45:31.832041 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd97061-80a5-4882-a153-a8e00df648da admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:31.848290 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd97061-80a5-4882-a153-a8e00df648da admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:31.853974 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180935a9-1f42-4515-89cf-b46f3414aee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:31.854622 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180935a9-1f42-4515-89cf-b46f3414aee2 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:31.854900 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180935a9-1f42-4515-89cf-b46f3414aee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:31.855205 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180935a9-1f42-4515-89cf-b46f3414aee2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:31.855874 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180935a9-1f42-4515-89cf-b46f3414aee2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:31.857552 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180935a9-1f42-4515-89cf-b46f3414aee2 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:31.868706 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 955/1923] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:31 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:45:31.873442 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4158f7-3b0d-4b2b-97cf-07f48e9b6d91 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:31.887632 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4158f7-3b0d-4b2b-97cf-07f48e9b6d91 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:31.896685 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe9de72-8e1b-4cf6-9496-9775943470fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:31.897676 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe9de72-8e1b-4cf6-9496-9775943470fd admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:31.898188 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe9de72-8e1b-4cf6-9496-9775943470fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:31.898696 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe9de72-8e1b-4cf6-9496-9775943470fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:31.899931 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe9de72-8e1b-4cf6-9496-9775943470fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:31.903074 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe9de72-8e1b-4cf6-9496-9775943470fd admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:31.921434 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 969/1924] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:31 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:31.926610 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180935a9-1f42-4515-89cf-b46f3414aee2 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:31.942159 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180935a9-1f42-4515-89cf-b46f3414aee2 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:31.950522 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52a2c94-a2ea-447b-907d-ea9e846d8cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:31.951047 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a2c94-a2ea-447b-907d-ea9e846d8cfd admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:31.951676 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a2c94-a2ea-447b-907d-ea9e846d8cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:31.951676 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a2c94-a2ea-447b-907d-ea9e846d8cfd admin admin] PATH_INFO: `/v3/projects/ca0a2572047a47a1802c7e6d699da73c/users/508b7cc4ee7f47babd61d991ae277c40/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:31.978292 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52a2c94-a2ea-447b-907d-ea9e846d8cfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca0a2572047a47a1802c7e6d699da73c, user_id=508b7cc4ee7f47babd61d991ae277c40, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:31.982313 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52a2c94-a2ea-447b-907d-ea9e846d8cfd admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:32.120877 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:32.128569 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 956/1925] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:31 2022] PUT /identity/v3/projects/ca0a2572047a47a1802c7e6d699da73c/users/508b7cc4ee7f47babd61d991ae277c40/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:32.141311 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee354c0e-8f24-47fe-bc11-b38e1f033498 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:32.141602 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee354c0e-8f24-47fe-bc11-b38e1f033498 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:32.141936 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee354c0e-8f24-47fe-bc11-b38e1f033498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:32.174691 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-ee354c0e-8f24-47fe-bc11-b38e1f033498 None None] MFA Rules not processed for user `508b7cc4ee7f47babd61d991ae277c40`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:32.176473 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-ee354c0e-8f24-47fe-bc11-b38e1f033498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:32.227784 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52a2c94-a2ea-447b-907d-ea9e846d8cfd admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:32.230970 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-c52a2c94-a2ea-447b-907d-ea9e846d8cfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:32.264453 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 970/1926] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:32 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:32.269959 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee354c0e-8f24-47fe-bc11-b38e1f033498 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:32.310853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52a2c94-a2ea-447b-907d-ea9e846d8cfd admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:32.317505 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1bd923-6ed8-4b63-948b-42033637fc85 tempest-VolumeQuotaClassesTest-1893277907 tempest-VolumeQuotaClassesTest-1893277907-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '961b21e3f0bc4121ae84b9ddcec2d57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8550878d05a349fd9ac882ed7848b606', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:32.318351 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bd923-6ed8-4b63-948b-42033637fc85 tempest-VolumeQuotaClassesTest-1893277907 tempest-VolumeQuotaClassesTest-1893277907-project-admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:32.318700 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bd923-6ed8-4b63-948b-42033637fc85 tempest-VolumeQuotaClassesTest-1893277907 tempest-VolumeQuotaClassesTest-1893277907-project-admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:32.319087 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bd923-6ed8-4b63-948b-42033637fc85 tempest-VolumeQuotaClassesTest-1893277907 tempest-VolumeQuotaClassesTest-1893277907-project-admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:32.320049 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1bd923-6ed8-4b63-948b-42033637fc85 tempest-VolumeQuotaClassesTest-1893277907 tempest-VolumeQuotaClassesTest-1893277907-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:45:32.320477 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1bd923-6ed8-4b63-948b-42033637fc85 tempest-VolumeQuotaClassesTest-1893277907 tempest-VolumeQuotaClassesTest-1893277907-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:32.322528 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1bd923-6ed8-4b63-948b-42033637fc85 tempest-VolumeQuotaClassesTest-1893277907 tempest-VolumeQuotaClassesTest-1893277907-project-admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:32.341403 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee354c0e-8f24-47fe-bc11-b38e1f033498 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:32.342426 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 957/1927] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:45:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:45:32.346754 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1bd923-6ed8-4b63-948b-42033637fc85 tempest-VolumeQuotaClassesTest-1893277907 tempest-VolumeQuotaClassesTest-1893277907-project-admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:32.347245 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f7c75-3d2e-48d9-be46-f18f8e3d88f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:32.347885 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f7c75-3d2e-48d9-be46-f18f8e3d88f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:32.348174 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f7c75-3d2e-48d9-be46-f18f8e3d88f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:32.348486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f7c75-3d2e-48d9-be46-f18f8e3d88f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:32.349131 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0f7c75-3d2e-48d9-be46-f18f8e3d88f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:32.351189 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0f7c75-3d2e-48d9-be46-f18f8e3d88f5 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:32.356744 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1bd923-6ed8-4b63-948b-42033637fc85 tempest-VolumeQuotaClassesTest-1893277907 tempest-VolumeQuotaClassesTest-1893277907-project-admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:32.362120 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e30108b-c0e7-484a-b0de-d00fdf991df6 tempest-VolumeQuotaClassesTest-1893277907 tempest-VolumeQuotaClassesTest-1893277907-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '961b21e3f0bc4121ae84b9ddcec2d57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8550878d05a349fd9ac882ed7848b606', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:32.362951 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e30108b-c0e7-484a-b0de-d00fdf991df6 tempest-VolumeQuotaClassesTest-1893277907 tempest-VolumeQuotaClassesTest-1893277907-project-admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:32.363291 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e30108b-c0e7-484a-b0de-d00fdf991df6 tempest-VolumeQuotaClassesTest-1893277907 tempest-VolumeQuotaClassesTest-1893277907-project-admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:32.363568 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e30108b-c0e7-484a-b0de-d00fdf991df6 tempest-VolumeQuotaClassesTest-1893277907 tempest-VolumeQuotaClassesTest-1893277907-project-admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:32.364195 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e30108b-c0e7-484a-b0de-d00fdf991df6 tempest-VolumeQuotaClassesTest-1893277907 tempest-VolumeQuotaClassesTest-1893277907-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:32.366427 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e30108b-c0e7-484a-b0de-d00fdf991df6 tempest-VolumeQuotaClassesTest-1893277907 tempest-VolumeQuotaClassesTest-1893277907-project-admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:32.551510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 971/1928] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:45:32 2022] POST /identity/v3/projects => generated 385 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:45:32.555516 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f7c75-3d2e-48d9-be46-f18f8e3d88f5 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:32.578234 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 958/1929] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:45:32 2022] POST /identity/v3/projects => generated 353 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:45:32.582863 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e30108b-c0e7-484a-b0de-d00fdf991df6 tempest-VolumeQuotaClassesTest-1893277907 tempest-VolumeQuotaClassesTest-1893277907-project-admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:32.633176 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f7c75-3d2e-48d9-be46-f18f8e3d88f5 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:32.653037 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e30108b-c0e7-484a-b0de-d00fdf991df6 tempest-VolumeQuotaClassesTest-1893277907 tempest-VolumeQuotaClassesTest-1893277907-project-admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:32.659254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f56898e-b84a-49b4-ac0c-be4aded74f87 tempest-VolumeQuotaClassesTest-1893277907 tempest-VolumeQuotaClassesTest-1893277907-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '961b21e3f0bc4121ae84b9ddcec2d57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8550878d05a349fd9ac882ed7848b606', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:32.659977 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f56898e-b84a-49b4-ac0c-be4aded74f87 tempest-VolumeQuotaClassesTest-1893277907 tempest-VolumeQuotaClassesTest-1893277907-project-admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:32.660420 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f56898e-b84a-49b4-ac0c-be4aded74f87 tempest-VolumeQuotaClassesTest-1893277907 tempest-VolumeQuotaClassesTest-1893277907-project-admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:32.660793 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f56898e-b84a-49b4-ac0c-be4aded74f87 tempest-VolumeQuotaClassesTest-1893277907 tempest-VolumeQuotaClassesTest-1893277907-project-admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:32.661548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f56898e-b84a-49b4-ac0c-be4aded74f87 tempest-VolumeQuotaClassesTest-1893277907 tempest-VolumeQuotaClassesTest-1893277907-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:45:32.661951 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f56898e-b84a-49b4-ac0c-be4aded74f87 tempest-VolumeQuotaClassesTest-1893277907 tempest-VolumeQuotaClassesTest-1893277907-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:32.663724 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f56898e-b84a-49b4-ac0c-be4aded74f87 tempest-VolumeQuotaClassesTest-1893277907 tempest-VolumeQuotaClassesTest-1893277907-project-admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:32.679504 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 959/1930] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:45:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:45:32.719778 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205af306-019e-4471-a16c-a6ee66b4870c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:32.720590 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205af306-019e-4471-a16c-a6ee66b4870c admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:32.721338 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205af306-019e-4471-a16c-a6ee66b4870c admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:32.721763 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205af306-019e-4471-a16c-a6ee66b4870c admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:32.723135 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205af306-019e-4471-a16c-a6ee66b4870c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:32.726750 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205af306-019e-4471-a16c-a6ee66b4870c admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:32.758220 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f56898e-b84a-49b4-ac0c-be4aded74f87 tempest-VolumeQuotaClassesTest-1893277907 tempest-VolumeQuotaClassesTest-1893277907-project-admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:32.776154 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f56898e-b84a-49b4-ac0c-be4aded74f87 tempest-VolumeQuotaClassesTest-1893277907 tempest-VolumeQuotaClassesTest-1893277907-project-admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:32.784228 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21536fde-9ebc-4648-9c1e-0e6944265575 tempest-VolumeQuotaClassesTest-1893277907 tempest-VolumeQuotaClassesTest-1893277907-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '961b21e3f0bc4121ae84b9ddcec2d57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8550878d05a349fd9ac882ed7848b606', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:32.785134 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21536fde-9ebc-4648-9c1e-0e6944265575 tempest-VolumeQuotaClassesTest-1893277907 tempest-VolumeQuotaClassesTest-1893277907-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:32.785598 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21536fde-9ebc-4648-9c1e-0e6944265575 tempest-VolumeQuotaClassesTest-1893277907 tempest-VolumeQuotaClassesTest-1893277907-project-admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:32.786077 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21536fde-9ebc-4648-9c1e-0e6944265575 tempest-VolumeQuotaClassesTest-1893277907 tempest-VolumeQuotaClassesTest-1893277907-project-admin] PATH_INFO: `/v3/projects/d1875cef4f5040d6b9540504131eb3c4` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:32.787845 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21536fde-9ebc-4648-9c1e-0e6944265575 tempest-VolumeQuotaClassesTest-1893277907 tempest-VolumeQuotaClassesTest-1893277907-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d1875cef4f5040d6b9540504131eb3c4)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:32.791360 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21536fde-9ebc-4648-9c1e-0e6944265575 tempest-VolumeQuotaClassesTest-1893277907 tempest-VolumeQuotaClassesTest-1893277907-project-admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:32.826556 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 972/1931] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:45:32 2022] POST /identity/v3/users => generated 299 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:32.828644 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205af306-019e-4471-a16c-a6ee66b4870c admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:32.846702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205af306-019e-4471-a16c-a6ee66b4870c admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:32.853363 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d865181e-7f18-4b2a-a57d-73e866c25386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:32.853995 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d865181e-7f18-4b2a-a57d-73e866c25386 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:32.854279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d865181e-7f18-4b2a-a57d-73e866c25386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:32.855997 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d865181e-7f18-4b2a-a57d-73e866c25386 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:32.855997 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d865181e-7f18-4b2a-a57d-73e866c25386 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:32.857120 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d865181e-7f18-4b2a-a57d-73e866c25386 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:32.874021 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 973/1932] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:45:32.875198 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d865181e-7f18-4b2a-a57d-73e866c25386 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:32.889106 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d865181e-7f18-4b2a-a57d-73e866c25386 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:32.894783 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b9bb87-4304-48d7-8881-eb33a0146dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:32.895476 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9bb87-4304-48d7-8881-eb33a0146dac admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:32.895753 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9bb87-4304-48d7-8881-eb33a0146dac admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:32.896029 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9bb87-4304-48d7-8881-eb33a0146dac admin admin] PATH_INFO: `/v3/projects/f4181d94648a4569899d6029f7e578f6/users/bee4d37ec26a4ec1bbf8c40bb130c00b/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:32.916806 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b9bb87-4304-48d7-8881-eb33a0146dac admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4181d94648a4569899d6029f7e578f6, user_id=bee4d37ec26a4ec1bbf8c40bb130c00b, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:32.919593 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b9bb87-4304-48d7-8881-eb33a0146dac admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:33.031409 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 974/1933] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:32 2022] PUT /identity/v3/projects/f4181d94648a4569899d6029f7e578f6/users/bee4d37ec26a4ec1bbf8c40bb130c00b/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:45:33.036261 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b9bb87-4304-48d7-8881-eb33a0146dac admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:33.050564 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-21536fde-9ebc-4648-9c1e-0e6944265575 tempest-VolumeQuotaClassesTest-1893277907 tempest-VolumeQuotaClassesTest-1893277907-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1875cef4f5040d6b9540504131eb3c4', 'request_id': 'req-21536fde-9ebc-4648-9c1e-0e6944265575'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:33.058337 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-21536fde-9ebc-4648-9c1e-0e6944265575 tempest-VolumeQuotaClassesTest-1893277907 tempest-VolumeQuotaClassesTest-1893277907-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1875cef4f5040d6b9540504131eb3c4', 'request_id': 'req-21536fde-9ebc-4648-9c1e-0e6944265575'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:33.120104 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b9bb87-4304-48d7-8881-eb33a0146dac admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:33.138837 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-21536fde-9ebc-4648-9c1e-0e6944265575 tempest-VolumeQuotaClassesTest-1893277907 tempest-VolumeQuotaClassesTest-1893277907-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1875cef4f5040d6b9540504131eb3c4', 'request_id': 'req-21536fde-9ebc-4648-9c1e-0e6944265575'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:33.152693 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:33.175808 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-21536fde-9ebc-4648-9c1e-0e6944265575 tempest-VolumeQuotaClassesTest-1893277907 tempest-VolumeQuotaClassesTest-1893277907-project-admin] The token cache is being invalidate because project d1875cef4f5040d6b9540504131eb3c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:45:33.176690 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-21536fde-9ebc-4648-9c1e-0e6944265575 tempest-VolumeQuotaClassesTest-1893277907 tempest-VolumeQuotaClassesTest-1893277907-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:33.179697 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-21536fde-9ebc-4648-9c1e-0e6944265575 tempest-VolumeQuotaClassesTest-1893277907 tempest-VolumeQuotaClassesTest-1893277907-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:33.182774 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 960/1934] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:45:32 2022] DELETE /identity/v3/projects/d1875cef4f5040d6b9540504131eb3c4 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:33.212651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea7377-a97f-43e4-9e42-a9aa3813d43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:33.213324 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea7377-a97f-43e4-9e42-a9aa3813d43f admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:33.213612 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea7377-a97f-43e4-9e42-a9aa3813d43f admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:33.213941 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea7377-a97f-43e4-9e42-a9aa3813d43f admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:33.214594 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bea7377-a97f-43e4-9e42-a9aa3813d43f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:33.216279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bea7377-a97f-43e4-9e42-a9aa3813d43f admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:33.232085 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:33.233217 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 975/1935] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:33 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:33.236924 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21536fde-9ebc-4648-9c1e-0e6944265575 tempest-VolumeQuotaClassesTest-1893277907 tempest-VolumeQuotaClassesTest-1893277907-project-admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:33.237056 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-21536fde-9ebc-4648-9c1e-0e6944265575 tempest-VolumeQuotaClassesTest-1893277907 tempest-VolumeQuotaClassesTest-1893277907-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:33.256434 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21536fde-9ebc-4648-9c1e-0e6944265575 tempest-VolumeQuotaClassesTest-1893277907 tempest-VolumeQuotaClassesTest-1893277907-project-admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:33.262604 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f74032-245a-43fe-81eb-4c0462a916cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:33.265034 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f74032-245a-43fe-81eb-4c0462a916cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:33.265034 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f74032-245a-43fe-81eb-4c0462a916cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:33.265034 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f74032-245a-43fe-81eb-4c0462a916cd admin admin] PATH_INFO: `/v3/projects/f4181d94648a4569899d6029f7e578f6/users/bee4d37ec26a4ec1bbf8c40bb130c00b/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:33.266161 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f74032-245a-43fe-81eb-4c0462a916cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4181d94648a4569899d6029f7e578f6, user_id=bee4d37ec26a4ec1bbf8c40bb130c00b, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:33.270120 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f74032-245a-43fe-81eb-4c0462a916cd admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:33.282321 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 961/1936] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:33 2022] PUT /identity/v3/projects/f4181d94648a4569899d6029f7e578f6/users/bee4d37ec26a4ec1bbf8c40bb130c00b/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:33.293990 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b12a38-9d8c-43ef-92d4-f189bdc49b37 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:33.294277 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b12a38-9d8c-43ef-92d4-f189bdc49b37 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:33.294554 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b12a38-9d8c-43ef-92d4-f189bdc49b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:33.334693 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-95b12a38-9d8c-43ef-92d4-f189bdc49b37 None None] MFA Rules not processed for user `bee4d37ec26a4ec1bbf8c40bb130c00b`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:33.336207 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-95b12a38-9d8c-43ef-92d4-f189bdc49b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:33.418250 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 976/1937] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:33 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:33.429510 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f74032-245a-43fe-81eb-4c0462a916cd admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:33.431195 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-91f74032-245a-43fe-81eb-4c0462a916cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:33.490088 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f74032-245a-43fe-81eb-4c0462a916cd admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:33.493756 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b823a-a5e8-4837-8238-dc6c20bb70e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:33.494362 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b823a-a5e8-4837-8238-dc6c20bb70e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:33.494646 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b823a-a5e8-4837-8238-dc6c20bb70e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:33.494925 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b823a-a5e8-4837-8238-dc6c20bb70e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:33.505304 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b823a-a5e8-4837-8238-dc6c20bb70e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:33.506777 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b823a-a5e8-4837-8238-dc6c20bb70e5 service cinder] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:33.516473 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 962/1938] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:33 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:45:33.801224 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:34.034955 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:34.109113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:34.191458 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b12a38-9d8c-43ef-92d4-f189bdc49b37 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:34.193559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-95b12a38-9d8c-43ef-92d4-f189bdc49b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:34.264704 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b12a38-9d8c-43ef-92d4-f189bdc49b37 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:34.270833 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b74ece-071c-4af3-8676-e26bc4da80cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:34.271575 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b74ece-071c-4af3-8676-e26bc4da80cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:34.272070 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b74ece-071c-4af3-8676-e26bc4da80cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:34.272522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b74ece-071c-4af3-8676-e26bc4da80cb admin admin] PATH_INFO: `/v3/users/6ead5bd6b21b46ab9b839fdef9be99ef` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:34.275553 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b74ece-071c-4af3-8676-e26bc4da80cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ead5bd6b21b46ab9b839fdef9be99ef)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:34.278204 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b74ece-071c-4af3-8676-e26bc4da80cb admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:34.503183 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:34.546694 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-36b74ece-071c-4af3-8676-e26bc4da80cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ead5bd6b21b46ab9b839fdef9be99ef'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:34.594292 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:34.652171 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-36b74ece-071c-4af3-8676-e26bc4da80cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ead5bd6b21b46ab9b839fdef9be99ef'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:34.661793 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-36b74ece-071c-4af3-8676-e26bc4da80cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ead5bd6b21b46ab9b839fdef9be99ef'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:34.675620 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-36b74ece-071c-4af3-8676-e26bc4da80cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ead5bd6b21b46ab9b839fdef9be99ef'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:34.676389 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-36b74ece-071c-4af3-8676-e26bc4da80cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ead5bd6b21b46ab9b839fdef9be99ef'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:34.682755 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 977/1939] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:45:34 2022] DELETE /identity/v3/users/6ead5bd6b21b46ab9b839fdef9be99ef => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:34.687492 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b3a2f-31fe-40b2-941c-9e2e2fa18dd3 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:34.687863 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b3a2f-31fe-40b2-941c-9e2e2fa18dd3 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:34.688201 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b3a2f-31fe-40b2-941c-9e2e2fa18dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:34.710270 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-6e2b3a2f-31fe-40b2-941c-9e2e2fa18dd3 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:34.711533 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-6e2b3a2f-31fe-40b2-941c-9e2e2fa18dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:34.781470 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 963/1940] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:34.788730 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b74ece-071c-4af3-8676-e26bc4da80cb admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:34.790174 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-36b74ece-071c-4af3-8676-e26bc4da80cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:34.848732 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b74ece-071c-4af3-8676-e26bc4da80cb admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:34.852411 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b7404-160e-42a4-a4c8-902be4ac1219 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:34.853036 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b7404-160e-42a4-a4c8-902be4ac1219 service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:34.853318 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b7404-160e-42a4-a4c8-902be4ac1219 service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:34.853592 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b7404-160e-42a4-a4c8-902be4ac1219 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:34.863583 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5b7404-160e-42a4-a4c8-902be4ac1219 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:34.865064 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5b7404-160e-42a4-a4c8-902be4ac1219 service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:34.875089 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 978/1941] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:45:35.146135 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2b3a2f-31fe-40b2-941c-9e2e2fa18dd3 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:35.147713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-6e2b3a2f-31fe-40b2-941c-9e2e2fa18dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:35.167044 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2b3a2f-31fe-40b2-941c-9e2e2fa18dd3 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:35.175601 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d1d37-4da8-488b-ba00-2e12c5ac7d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:35.177031 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d1d37-4da8-488b-ba00-2e12c5ac7d99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:35.177527 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d1d37-4da8-488b-ba00-2e12c5ac7d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:35.178050 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d1d37-4da8-488b-ba00-2e12c5ac7d99 admin admin] PATH_INFO: `/v3/users/508b7cc4ee7f47babd61d991ae277c40` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:35.181353 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0d1d37-4da8-488b-ba00-2e12c5ac7d99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=508b7cc4ee7f47babd61d991ae277c40)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:35.182950 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0d1d37-4da8-488b-ba00-2e12c5ac7d99 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:35.210380 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:35.476570 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b7404-160e-42a4-a4c8-902be4ac1219 service neutron] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:35.478086 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-ab5b7404-160e-42a4-a4c8-902be4ac1219 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:35.492783 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b7404-160e-42a4-a4c8-902be4ac1219 service neutron] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:35.505861 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:35.507442 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85073198-4213-40ed-9455-e6e9bf187fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:35.508360 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85073198-4213-40ed-9455-e6e9bf187fc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:35.508714 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85073198-4213-40ed-9455-e6e9bf187fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:35.509059 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85073198-4213-40ed-9455-e6e9bf187fc4 admin admin] PATH_INFO: `/v3/projects/b09f9c6ff4e24b16be8d00931698b465` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:35.510298 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85073198-4213-40ed-9455-e6e9bf187fc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b09f9c6ff4e24b16be8d00931698b465)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:35.514723 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85073198-4213-40ed-9455-e6e9bf187fc4 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:35.526663 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-2c0d1d37-4da8-488b-ba00-2e12c5ac7d99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '508b7cc4ee7f47babd61d991ae277c40'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:35.535116 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-2c0d1d37-4da8-488b-ba00-2e12c5ac7d99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '508b7cc4ee7f47babd61d991ae277c40'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:35.535956 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-2c0d1d37-4da8-488b-ba00-2e12c5ac7d99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '508b7cc4ee7f47babd61d991ae277c40'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:35.668545 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-2c0d1d37-4da8-488b-ba00-2e12c5ac7d99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '508b7cc4ee7f47babd61d991ae277c40'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:35.683182 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-2c0d1d37-4da8-488b-ba00-2e12c5ac7d99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '508b7cc4ee7f47babd61d991ae277c40'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:35.689903 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 964/1942] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:45:35 2022] DELETE /identity/v3/users/508b7cc4ee7f47babd61d991ae277c40 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:35.696166 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85318b-aa3c-4228-b321-62dc1189376e None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:35.696797 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85318b-aa3c-4228-b321-62dc1189376e None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:35.697314 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85318b-aa3c-4228-b321-62dc1189376e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:35.731063 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-85073198-4213-40ed-9455-e6e9bf187fc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b09f9c6ff4e24b16be8d00931698b465', 'request_id': 'req-85073198-4213-40ed-9455-e6e9bf187fc4'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:35.732108 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-5f85318b-aa3c-4228-b321-62dc1189376e None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:35.734416 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-5f85318b-aa3c-4228-b321-62dc1189376e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:35.740794 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-85073198-4213-40ed-9455-e6e9bf187fc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b09f9c6ff4e24b16be8d00931698b465', 'request_id': 'req-85073198-4213-40ed-9455-e6e9bf187fc4'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:35.820103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 965/1943] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:35.833342 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f85318b-aa3c-4228-b321-62dc1189376e None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:35.833342 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-5f85318b-aa3c-4228-b321-62dc1189376e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:35.852657 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-85073198-4213-40ed-9455-e6e9bf187fc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b09f9c6ff4e24b16be8d00931698b465', 'request_id': 'req-85073198-4213-40ed-9455-e6e9bf187fc4'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:35.871397 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-85073198-4213-40ed-9455-e6e9bf187fc4 admin admin] The token cache is being invalidate because project b09f9c6ff4e24b16be8d00931698b465 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:45:35.871694 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-85073198-4213-40ed-9455-e6e9bf187fc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:35.872694 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-85073198-4213-40ed-9455-e6e9bf187fc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:35.876438 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 979/1944] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:45:35 2022] DELETE /identity/v3/projects/b09f9c6ff4e24b16be8d00931698b465 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:35.879228 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85073198-4213-40ed-9455-e6e9bf187fc4 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:35.881447 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-85073198-4213-40ed-9455-e6e9bf187fc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:35.905884 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f85318b-aa3c-4228-b321-62dc1189376e None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:35.911905 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85073198-4213-40ed-9455-e6e9bf187fc4 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:35.913447 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a53bb3e-4a53-4ca0-98c6-876562e3ad39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:35.914702 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a53bb3e-4a53-4ca0-98c6-876562e3ad39 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:35.915174 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a53bb3e-4a53-4ca0-98c6-876562e3ad39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:35.915643 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a53bb3e-4a53-4ca0-98c6-876562e3ad39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:35.918769 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-0a53bb3e-4a53-4ca0-98c6-876562e3ad39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:35.919992 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2164f1a3-9551-4d45-b653-4d362fe6d0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:35.921219 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164f1a3-9551-4d45-b653-4d362fe6d0a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:35.921724 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164f1a3-9551-4d45-b653-4d362fe6d0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:35.922315 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164f1a3-9551-4d45-b653-4d362fe6d0a5 admin admin] PATH_INFO: `/v3/users/961b21e3f0bc4121ae84b9ddcec2d57e` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:35.925815 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2164f1a3-9551-4d45-b653-4d362fe6d0a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=961b21e3f0bc4121ae84b9ddcec2d57e)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:35.929764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2164f1a3-9551-4d45-b653-4d362fe6d0a5 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:35.939154 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a53bb3e-4a53-4ca0-98c6-876562e3ad39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:35.943454 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a53bb3e-4a53-4ca0-98c6-876562e3ad39 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:35.962293 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 966/1945] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:45:36.159737 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-2164f1a3-9551-4d45-b653-4d362fe6d0a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '961b21e3f0bc4121ae84b9ddcec2d57e'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:36.628094 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-2164f1a3-9551-4d45-b653-4d362fe6d0a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '961b21e3f0bc4121ae84b9ddcec2d57e'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:36.628094 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-2164f1a3-9551-4d45-b653-4d362fe6d0a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '961b21e3f0bc4121ae84b9ddcec2d57e'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:36.628094 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-2164f1a3-9551-4d45-b653-4d362fe6d0a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '961b21e3f0bc4121ae84b9ddcec2d57e'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:36.628094 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-2164f1a3-9551-4d45-b653-4d362fe6d0a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '961b21e3f0bc4121ae84b9ddcec2d57e'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:36.628094 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 980/1946] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:45:35 2022] DELETE /identity/v3/users/961b21e3f0bc4121ae84b9ddcec2d57e => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:45:36.862011 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a53bb3e-4a53-4ca0-98c6-876562e3ad39 service neutron] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:36.863481 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-0a53bb3e-4a53-4ca0-98c6-876562e3ad39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:36.876827 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a53bb3e-4a53-4ca0-98c6-876562e3ad39 service neutron] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:36.975160 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af86fd02-5f9c-40df-9a5c-b6ffe0eb7440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:36.975160 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86fd02-5f9c-40df-9a5c-b6ffe0eb7440 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:36.975160 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86fd02-5f9c-40df-9a5c-b6ffe0eb7440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:36.975160 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86fd02-5f9c-40df-9a5c-b6ffe0eb7440 admin admin] PATH_INFO: `/v3/projects/ca0a2572047a47a1802c7e6d699da73c` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:36.976337 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af86fd02-5f9c-40df-9a5c-b6ffe0eb7440 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca0a2572047a47a1802c7e6d699da73c)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:36.976337 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af86fd02-5f9c-40df-9a5c-b6ffe0eb7440 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:37.259213 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:37.396380 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2164f1a3-9551-4d45-b653-4d362fe6d0a5 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:37.397937 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-2164f1a3-9551-4d45-b653-4d362fe6d0a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:37.459529 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2164f1a3-9551-4d45-b653-4d362fe6d0a5 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:37.463232 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06861898-b985-4473-b469-e6501d29967e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:37.463860 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06861898-b985-4473-b469-e6501d29967e service cinder] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:37.464140 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06861898-b985-4473-b469-e6501d29967e service cinder] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:37.464446 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06861898-b985-4473-b469-e6501d29967e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:37.466339 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-06861898-b985-4473-b469-e6501d29967e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:37.532814 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06861898-b985-4473-b469-e6501d29967e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:37.534410 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06861898-b985-4473-b469-e6501d29967e service cinder] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:37.557529 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 981/1947] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:45:37.574004 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-af86fd02-5f9c-40df-9a5c-b6ffe0eb7440 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca0a2572047a47a1802c7e6d699da73c', 'request_id': 'req-af86fd02-5f9c-40df-9a5c-b6ffe0eb7440'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:37.582184 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-af86fd02-5f9c-40df-9a5c-b6ffe0eb7440 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca0a2572047a47a1802c7e6d699da73c', 'request_id': 'req-af86fd02-5f9c-40df-9a5c-b6ffe0eb7440'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:37.702944 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-af86fd02-5f9c-40df-9a5c-b6ffe0eb7440 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca0a2572047a47a1802c7e6d699da73c', 'request_id': 'req-af86fd02-5f9c-40df-9a5c-b6ffe0eb7440'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:37.722875 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-af86fd02-5f9c-40df-9a5c-b6ffe0eb7440 admin admin] The token cache is being invalidate because project ca0a2572047a47a1802c7e6d699da73c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:45:37.722875 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-af86fd02-5f9c-40df-9a5c-b6ffe0eb7440 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:37.723722 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-af86fd02-5f9c-40df-9a5c-b6ffe0eb7440 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:37.726732 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 967/1948] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:45:36 2022] DELETE /identity/v3/projects/ca0a2572047a47a1802c7e6d699da73c => generated 0 bytes in 866 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:37.733521 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06861898-b985-4473-b469-e6501d29967e service cinder] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:37.737581 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-06861898-b985-4473-b469-e6501d29967e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:37.768988 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06861898-b985-4473-b469-e6501d29967e service cinder] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:37.894717 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271b05e-31da-4cae-b465-52d38f8168d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:37.895479 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271b05e-31da-4cae-b465-52d38f8168d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:37.895764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271b05e-31da-4cae-b465-52d38f8168d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:37.896043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271b05e-31da-4cae-b465-52d38f8168d0 admin admin] PATH_INFO: `/v3/users/bee4d37ec26a4ec1bbf8c40bb130c00b` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:37.897301 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c271b05e-31da-4cae-b465-52d38f8168d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bee4d37ec26a4ec1bbf8c40bb130c00b)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:37.899495 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c271b05e-31da-4cae-b465-52d38f8168d0 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:37.909253 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:37.919829 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af86fd02-5f9c-40df-9a5c-b6ffe0eb7440 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:37.921594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-af86fd02-5f9c-40df-9a5c-b6ffe0eb7440 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:37.950546 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af86fd02-5f9c-40df-9a5c-b6ffe0eb7440 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:37.981939 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c979d-084b-4523-ad95-a556fd4f2823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:37.982678 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c979d-084b-4523-ad95-a556fd4f2823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:37.982959 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c979d-084b-4523-ad95-a556fd4f2823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:37.983421 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c979d-084b-4523-ad95-a556fd4f2823 admin admin] PATH_INFO: `/v3/projects/8550878d05a349fd9ac882ed7848b606` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:37.985818 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c979d-084b-4523-ad95-a556fd4f2823 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8550878d05a349fd9ac882ed7848b606)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:37.989589 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c979d-084b-4523-ad95-a556fd4f2823 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:38.210266 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-d53c979d-084b-4523-ad95-a556fd4f2823 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8550878d05a349fd9ac882ed7848b606', 'request_id': 'req-d53c979d-084b-4523-ad95-a556fd4f2823'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:38.218326 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-c271b05e-31da-4cae-b465-52d38f8168d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bee4d37ec26a4ec1bbf8c40bb130c00b'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:38.223676 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-d53c979d-084b-4523-ad95-a556fd4f2823 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8550878d05a349fd9ac882ed7848b606', 'request_id': 'req-d53c979d-084b-4523-ad95-a556fd4f2823'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:38.360524 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-c271b05e-31da-4cae-b465-52d38f8168d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bee4d37ec26a4ec1bbf8c40bb130c00b'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:38.370835 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-c271b05e-31da-4cae-b465-52d38f8168d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bee4d37ec26a4ec1bbf8c40bb130c00b'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:38.384493 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-c271b05e-31da-4cae-b465-52d38f8168d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bee4d37ec26a4ec1bbf8c40bb130c00b'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:38.385305 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-c271b05e-31da-4cae-b465-52d38f8168d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bee4d37ec26a4ec1bbf8c40bb130c00b'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:38.393595 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 982/1949] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:45:37 2022] DELETE /identity/v3/users/bee4d37ec26a4ec1bbf8c40bb130c00b => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:45:38.424901 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:38.497770 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-d53c979d-084b-4523-ad95-a556fd4f2823 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8550878d05a349fd9ac882ed7848b606', 'request_id': 'req-d53c979d-084b-4523-ad95-a556fd4f2823'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:38.518324 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-d53c979d-084b-4523-ad95-a556fd4f2823 admin admin] The token cache is being invalidate because project 8550878d05a349fd9ac882ed7848b606 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:45:38.518611 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-d53c979d-084b-4523-ad95-a556fd4f2823 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:38.519287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-d53c979d-084b-4523-ad95-a556fd4f2823 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:38.520933 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 968/1950] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:45:37 2022] DELETE /identity/v3/projects/8550878d05a349fd9ac882ed7848b606 => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:38.525970 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d165d99-642a-4a4d-ac26-d4228d8d79ec None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:38.526376 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d165d99-642a-4a4d-ac26-d4228d8d79ec None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:38.526791 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d165d99-642a-4a4d-ac26-d4228d8d79ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:38.552562 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-1d165d99-642a-4a4d-ac26-d4228d8d79ec None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:38.554144 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-1d165d99-642a-4a4d-ac26-d4228d8d79ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:38.627778 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 983/1951] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:45:38.637204 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016ac5f-59dd-4c98-a3ca-aa0273887350 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:38.637560 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016ac5f-59dd-4c98-a3ca-aa0273887350 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:38.637916 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016ac5f-59dd-4c98-a3ca-aa0273887350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:38.681237 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-4016ac5f-59dd-4c98-a3ca-aa0273887350 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:38.684996 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-4016ac5f-59dd-4c98-a3ca-aa0273887350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:38.709639 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 969/1952] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:38.711696 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d165d99-642a-4a4d-ac26-d4228d8d79ec None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:38.728701 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d165d99-642a-4a4d-ac26-d4228d8d79ec None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:38.733642 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa8129-fc6b-4201-b471-cc470966a090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:38.734691 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa8129-fc6b-4201-b471-cc470966a090 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:38.734983 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa8129-fc6b-4201-b471-cc470966a090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:38.735259 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa8129-fc6b-4201-b471-cc470966a090 admin admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:38.735918 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa8129-fc6b-4201-b471-cc470966a090 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:45:38.737010 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa8129-fc6b-4201-b471-cc470966a090 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:38.738476 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa8129-fc6b-4201-b471-cc470966a090 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:38.751197 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 984/1953] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:45:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:45:38.756292 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4016ac5f-59dd-4c98-a3ca-aa0273887350 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:38.768793 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4016ac5f-59dd-4c98-a3ca-aa0273887350 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:38.774305 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3933eb-d096-4526-b246-3e47d50ee0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:38.775177 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3933eb-d096-4526-b246-3e47d50ee0bd admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:38.775591 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3933eb-d096-4526-b246-3e47d50ee0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:38.776388 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3933eb-d096-4526-b246-3e47d50ee0bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:38.777237 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3933eb-d096-4526-b246-3e47d50ee0bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:38.779053 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3933eb-d096-4526-b246-3e47d50ee0bd admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:38.996490 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 970/1954] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:45:38 2022] POST /identity/v3/projects => generated 385 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:45:38.998231 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa8129-fc6b-4201-b471-cc470966a090 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:39.031238 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa8129-fc6b-4201-b471-cc470966a090 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:39.115586 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a05c47-320e-485f-89d3-fb77c0e4223c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:39.116262 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a05c47-320e-485f-89d3-fb77c0e4223c admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:39.116810 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a05c47-320e-485f-89d3-fb77c0e4223c admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:39.117202 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a05c47-320e-485f-89d3-fb77c0e4223c admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:39.118072 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a05c47-320e-485f-89d3-fb77c0e4223c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:39.120043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a05c47-320e-485f-89d3-fb77c0e4223c admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:39.257292 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 985/1955] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:45:38 2022] POST /identity/v3/users => generated 293 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:39.259135 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3933eb-d096-4526-b246-3e47d50ee0bd admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:39.270814 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3933eb-d096-4526-b246-3e47d50ee0bd admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:39.276696 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a9babf-0c50-47c1-9264-a37a61382621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:39.277333 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9babf-0c50-47c1-9264-a37a61382621 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:39.277609 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9babf-0c50-47c1-9264-a37a61382621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:39.278140 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9babf-0c50-47c1-9264-a37a61382621 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:39.279052 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a9babf-0c50-47c1-9264-a37a61382621 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:39.281015 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a9babf-0c50-47c1-9264-a37a61382621 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:39.310349 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 971/1956] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:45:39.313039 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a05c47-320e-485f-89d3-fb77c0e4223c admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:39.353801 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a9babf-0c50-47c1-9264-a37a61382621 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:39.355972 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-30a9babf-0c50-47c1-9264-a37a61382621 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:39.359937 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a05c47-320e-485f-89d3-fb77c0e4223c admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:39.379144 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a9babf-0c50-47c1-9264-a37a61382621 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:39.386084 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df73b83d-710a-41ce-ac30-3dfbb181c7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:39.387132 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df73b83d-710a-41ce-ac30-3dfbb181c7e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:39.387628 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df73b83d-710a-41ce-ac30-3dfbb181c7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:39.388118 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df73b83d-710a-41ce-ac30-3dfbb181c7e7 admin admin] PATH_INFO: `/v3/projects/f4181d94648a4569899d6029f7e578f6` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:39.390189 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df73b83d-710a-41ce-ac30-3dfbb181c7e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4181d94648a4569899d6029f7e578f6)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:39.393221 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df73b83d-710a-41ce-ac30-3dfbb181c7e7 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:39.408423 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfb244b-7a07-4377-82a7-c6a716fbc388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:39.409088 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfb244b-7a07-4377-82a7-c6a716fbc388 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:39.413292 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfb244b-7a07-4377-82a7-c6a716fbc388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:39.413584 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfb244b-7a07-4377-82a7-c6a716fbc388 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:39.414295 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfb244b-7a07-4377-82a7-c6a716fbc388 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:39.423138 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfb244b-7a07-4377-82a7-c6a716fbc388 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:39.458133 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 986/1957] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:39.459832 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfb244b-7a07-4377-82a7-c6a716fbc388 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:39.481484 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfb244b-7a07-4377-82a7-c6a716fbc388 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:39.490040 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b1142-7523-4caa-927d-53efaee37725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:39.490772 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b1142-7523-4caa-927d-53efaee37725 admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:39.491066 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b1142-7523-4caa-927d-53efaee37725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:39.491349 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b1142-7523-4caa-927d-53efaee37725 admin admin] PATH_INFO: `/v3/projects/84a5522913ee448bb8593f90cb7af0f6/users/20757e37849240ef8f3209949a630117/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:39.535826 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1b1142-7523-4caa-927d-53efaee37725 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84a5522913ee448bb8593f90cb7af0f6, user_id=20757e37849240ef8f3209949a630117, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:39.538620 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1b1142-7523-4caa-927d-53efaee37725 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:39.543025 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-df73b83d-710a-41ce-ac30-3dfbb181c7e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4181d94648a4569899d6029f7e578f6', 'request_id': 'req-df73b83d-710a-41ce-ac30-3dfbb181c7e7'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:39.556171 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-df73b83d-710a-41ce-ac30-3dfbb181c7e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4181d94648a4569899d6029f7e578f6', 'request_id': 'req-df73b83d-710a-41ce-ac30-3dfbb181c7e7'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:39.644251 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 987/1958] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:39 2022] PUT /identity/v3/projects/84a5522913ee448bb8593f90cb7af0f6/users/20757e37849240ef8f3209949a630117/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:39.656898 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5444b4f-bcf7-4d5a-9b8c-b317ac6b4168 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:39.657412 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5444b4f-bcf7-4d5a-9b8c-b317ac6b4168 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:39.658465 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5444b4f-bcf7-4d5a-9b8c-b317ac6b4168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:39.677011 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-df73b83d-710a-41ce-ac30-3dfbb181c7e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4181d94648a4569899d6029f7e578f6', 'request_id': 'req-df73b83d-710a-41ce-ac30-3dfbb181c7e7'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:39.693571 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-df73b83d-710a-41ce-ac30-3dfbb181c7e7 admin admin] The token cache is being invalidate because project f4181d94648a4569899d6029f7e578f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:45:39.693957 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-df73b83d-710a-41ce-ac30-3dfbb181c7e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:39.694572 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-df73b83d-710a-41ce-ac30-3dfbb181c7e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:39.696440 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 972/1959] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:45:39 2022] DELETE /identity/v3/projects/f4181d94648a4569899d6029f7e578f6 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:39.703097 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43442fd9-bc4c-4364-9069-6ad3664c1464 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:39.703480 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-e5444b4f-bcf7-4d5a-9b8c-b317ac6b4168 None None] MFA Rules not processed for user `20757e37849240ef8f3209949a630117`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:39.703591 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43442fd9-bc4c-4364-9069-6ad3664c1464 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:39.703796 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43442fd9-bc4c-4364-9069-6ad3664c1464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:39.705889 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-e5444b4f-bcf7-4d5a-9b8c-b317ac6b4168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:39.746740 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-43442fd9-bc4c-4364-9069-6ad3664c1464 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:39.748061 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-43442fd9-bc4c-4364-9069-6ad3664c1464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:39.808826 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 988/1960] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:39.814084 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5444b4f-bcf7-4d5a-9b8c-b317ac6b4168 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:39.817045 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-e5444b4f-bcf7-4d5a-9b8c-b317ac6b4168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:39.859845 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 973/1961] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:45:39.870170 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a69c2-53c6-44f4-818c-3a9f486ff4f4 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:39.871523 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a69c2-53c6-44f4-818c-3a9f486ff4f4 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:39.872055 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a69c2-53c6-44f4-818c-3a9f486ff4f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:39.895398 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-972a69c2-53c6-44f4-818c-3a9f486ff4f4 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:39.897066 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-972a69c2-53c6-44f4-818c-3a9f486ff4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:39.912621 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 974/1962] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:39.917525 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972a69c2-53c6-44f4-818c-3a9f486ff4f4 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:39.929751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:39.935463 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972a69c2-53c6-44f4-818c-3a9f486ff4f4 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:39.935744 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5444b4f-bcf7-4d5a-9b8c-b317ac6b4168 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:39.936107 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9834d36a-4e59-4c19-bf62-e5ddd54da7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:39.936811 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834d36a-4e59-4c19-bf62-e5ddd54da7b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:39.937151 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834d36a-4e59-4c19-bf62-e5ddd54da7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:39.937431 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834d36a-4e59-4c19-bf62-e5ddd54da7b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:39.938093 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9834d36a-4e59-4c19-bf62-e5ddd54da7b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:45:39.938404 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9834d36a-4e59-4c19-bf62-e5ddd54da7b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:39.940562 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9834d36a-4e59-4c19-bf62-e5ddd54da7b7 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:39.945001 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ce549-098a-428d-a8ce-4fea74f18106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:39.945629 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ce549-098a-428d-a8ce-4fea74f18106 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:39.945915 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ce549-098a-428d-a8ce-4fea74f18106 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:39.946389 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ce549-098a-428d-a8ce-4fea74f18106 admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:39.947056 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911ce549-098a-428d-a8ce-4fea74f18106 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:39.950315 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911ce549-098a-428d-a8ce-4fea74f18106 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:39.954593 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 975/1963] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:45:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:45:39.960015 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9834d36a-4e59-4c19-bf62-e5ddd54da7b7 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:39.971833 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9834d36a-4e59-4c19-bf62-e5ddd54da7b7 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:39.977436 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec33ba2a-4bb7-4e59-b94b-a1f64c879f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:39.978129 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec33ba2a-4bb7-4e59-b94b-a1f64c879f41 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:39.978422 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec33ba2a-4bb7-4e59-b94b-a1f64c879f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:39.978798 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec33ba2a-4bb7-4e59-b94b-a1f64c879f41 admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:39.979810 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec33ba2a-4bb7-4e59-b94b-a1f64c879f41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:39.982109 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec33ba2a-4bb7-4e59-b94b-a1f64c879f41 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:40.110244 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 976/1964] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:45:39 2022] POST /identity/v3/projects => generated 359 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:45:40.111111 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 989/1965] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:45:39 2022] POST /identity/v3/projects => generated 387 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:40.113215 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec33ba2a-4bb7-4e59-b94b-a1f64c879f41 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:40.113385 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ce549-098a-428d-a8ce-4fea74f18106 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:40.240749 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec33ba2a-4bb7-4e59-b94b-a1f64c879f41 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:40.257981 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:40.269474 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ce549-098a-428d-a8ce-4fea74f18106 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:40.276572 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba290a-fa37-4eba-9e3a-64c9066be38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:40.277629 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba290a-fa37-4eba-9e3a-64c9066be38d admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:40.278218 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba290a-fa37-4eba-9e3a-64c9066be38d admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:40.278812 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba290a-fa37-4eba-9e3a-64c9066be38d admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:40.280081 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba290a-fa37-4eba-9e3a-64c9066be38d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:40.285248 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba290a-fa37-4eba-9e3a-64c9066be38d admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:40.341475 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed37791-edac-49cd-80e1-6b3262a3d8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:40.343119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed37791-edac-49cd-80e1-6b3262a3d8af admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:40.343119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed37791-edac-49cd-80e1-6b3262a3d8af admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:40.343754 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed37791-edac-49cd-80e1-6b3262a3d8af admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:40.345274 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed37791-edac-49cd-80e1-6b3262a3d8af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:40.352815 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed37791-edac-49cd-80e1-6b3262a3d8af admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:40.382118 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 990/1966] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:45:40 2022] POST /identity/v3/users => generated 294 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:40.384983 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba290a-fa37-4eba-9e3a-64c9066be38d admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:40.402791 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba290a-fa37-4eba-9e3a-64c9066be38d admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:40.432900 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5461bc-3350-4098-b3c4-1a8a7529fda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:40.432900 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5461bc-3350-4098-b3c4-1a8a7529fda4 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:40.432900 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5461bc-3350-4098-b3c4-1a8a7529fda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:40.432900 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5461bc-3350-4098-b3c4-1a8a7529fda4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:40.433329 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5461bc-3350-4098-b3c4-1a8a7529fda4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:40.434396 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5461bc-3350-4098-b3c4-1a8a7529fda4 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:40.465860 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 991/1967] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:40 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:45:40.467395 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5461bc-3350-4098-b3c4-1a8a7529fda4 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:40.485961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5461bc-3350-4098-b3c4-1a8a7529fda4 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:40.498046 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b95cec-67cd-42d8-bc6e-d09d8824524e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:40.498688 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b95cec-67cd-42d8-bc6e-d09d8824524e admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:40.498974 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b95cec-67cd-42d8-bc6e-d09d8824524e admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:40.499248 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b95cec-67cd-42d8-bc6e-d09d8824524e admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:40.499905 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b95cec-67cd-42d8-bc6e-d09d8824524e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:40.523038 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b95cec-67cd-42d8-bc6e-d09d8824524e admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:40.526125 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 977/1968] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:45:40 2022] POST /identity/v3/users => generated 280 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:40.528532 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed37791-edac-49cd-80e1-6b3262a3d8af admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:40.536237 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 992/1969] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:40 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:45:40.537205 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b95cec-67cd-42d8-bc6e-d09d8824524e admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:40.544567 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-3ed37791-edac-49cd-80e1-6b3262a3d8af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:40.559570 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b95cec-67cd-42d8-bc6e-d09d8824524e admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:40.580380 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f92d88-199d-42f3-a628-1bc4c476889c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:40.583866 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f92d88-199d-42f3-a628-1bc4c476889c admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:40.585244 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f92d88-199d-42f3-a628-1bc4c476889c admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:40.586509 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f92d88-199d-42f3-a628-1bc4c476889c admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:40.595743 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f92d88-199d-42f3-a628-1bc4c476889c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:40.601529 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f92d88-199d-42f3-a628-1bc4c476889c admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:40.628425 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 993/1970] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:40 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:40.633625 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f92d88-199d-42f3-a628-1bc4c476889c admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:40.671720 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f92d88-199d-42f3-a628-1bc4c476889c admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:40.683429 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934bc55f-6a40-451f-aaa2-cb4daac7f1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:40.684347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934bc55f-6a40-451f-aaa2-cb4daac7f1a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:40.684824 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934bc55f-6a40-451f-aaa2-cb4daac7f1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:40.685404 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934bc55f-6a40-451f-aaa2-cb4daac7f1a2 admin admin] PATH_INFO: `/v3/projects/417a2c54a3a048bcaa6cdc49fd08ada9/users/54afd1b4b1c64fb29c0c76b880c52ac3/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:40.719667 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed37791-edac-49cd-80e1-6b3262a3d8af admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:40.724807 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab137856-f5bf-4aae-b406-95d00dd44bb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:40.725564 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab137856-f5bf-4aae-b406-95d00dd44bb0 service cinder] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:40.725914 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab137856-f5bf-4aae-b406-95d00dd44bb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:40.726323 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab137856-f5bf-4aae-b406-95d00dd44bb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:40.727145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934bc55f-6a40-451f-aaa2-cb4daac7f1a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=417a2c54a3a048bcaa6cdc49fd08ada9, user_id=54afd1b4b1c64fb29c0c76b880c52ac3, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:40.728287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-ab137856-f5bf-4aae-b406-95d00dd44bb0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:40.729919 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934bc55f-6a40-451f-aaa2-cb4daac7f1a2 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:40.846257 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 994/1971] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:40 2022] PUT /identity/v3/projects/417a2c54a3a048bcaa6cdc49fd08ada9/users/54afd1b4b1c64fb29c0c76b880c52ac3/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:40.857053 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934bc55f-6a40-451f-aaa2-cb4daac7f1a2 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:40.874217 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab137856-f5bf-4aae-b406-95d00dd44bb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:40.875784 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab137856-f5bf-4aae-b406-95d00dd44bb0 service cinder] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:40.904900 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 978/1972] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:40 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:45:40.908439 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b40b921-692a-4647-abec-3bf06b7e4df5 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:40.908717 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b40b921-692a-4647-abec-3bf06b7e4df5 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:40.908975 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b40b921-692a-4647-abec-3bf06b7e4df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:40.967294 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-9b40b921-692a-4647-abec-3bf06b7e4df5 None None] MFA Rules not processed for user `54afd1b4b1c64fb29c0c76b880c52ac3`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:40.971292 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-9b40b921-692a-4647-abec-3bf06b7e4df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:40.988759 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:41.019084 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934bc55f-6a40-451f-aaa2-cb4daac7f1a2 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:41.027936 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb52f0-f0c8-4578-9fd8-ecc45ae72ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:41.028790 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb52f0-f0c8-4578-9fd8-ecc45ae72ab1 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:41.029080 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb52f0-f0c8-4578-9fd8-ecc45ae72ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:41.029360 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb52f0-f0c8-4578-9fd8-ecc45ae72ab1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:41.030027 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb52f0-f0c8-4578-9fd8-ecc45ae72ab1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:41.031763 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb52f0-f0c8-4578-9fd8-ecc45ae72ab1 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:41.055799 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 995/1973] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:40 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:41.065816 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb52f0-f0c8-4578-9fd8-ecc45ae72ab1 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:41.080984 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb52f0-f0c8-4578-9fd8-ecc45ae72ab1 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:41.093320 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e67d11-d6fe-4d3c-839a-84393f97aac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:41.095013 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e67d11-d6fe-4d3c-839a-84393f97aac4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:41.095737 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 979/1974] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:40 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:41.097764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e67d11-d6fe-4d3c-839a-84393f97aac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:41.097764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e67d11-d6fe-4d3c-839a-84393f97aac4 admin admin] PATH_INFO: `/v3/projects/c1b71be11673464e8f76018ba6ec26c6/users/de54188935dd430d953ca5c7698b47f2/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:41.101235 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b40b921-692a-4647-abec-3bf06b7e4df5 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:41.124263 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b40b921-692a-4647-abec-3bf06b7e4df5 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:41.132452 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e67d11-d6fe-4d3c-839a-84393f97aac4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1b71be11673464e8f76018ba6ec26c6, user_id=de54188935dd430d953ca5c7698b47f2, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:41.142162 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e67d11-d6fe-4d3c-839a-84393f97aac4 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:41.142395 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e79208-6593-4270-8cc3-e0448ad70837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:41.142395 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e79208-6593-4270-8cc3-e0448ad70837 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:41.142395 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e79208-6593-4270-8cc3-e0448ad70837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:41.142395 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e79208-6593-4270-8cc3-e0448ad70837 admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:41.142780 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e79208-6593-4270-8cc3-e0448ad70837 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:41.143032 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e79208-6593-4270-8cc3-e0448ad70837 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:41.287767 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 996/1975] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:41 2022] PUT /identity/v3/projects/c1b71be11673464e8f76018ba6ec26c6/users/de54188935dd430d953ca5c7698b47f2/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:41.299768 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f4c80f-4657-4b4b-8369-7ee64a16e009 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:41.300128 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f4c80f-4657-4b4b-8369-7ee64a16e009 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:41.300626 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f4c80f-4657-4b4b-8369-7ee64a16e009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:41.329380 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 980/1976] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:45:41 2022] POST /identity/v3/projects => generated 387 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:41.336473 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e79208-6593-4270-8cc3-e0448ad70837 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:41.374664 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-b3f4c80f-4657-4b4b-8369-7ee64a16e009 None None] MFA Rules not processed for user `de54188935dd430d953ca5c7698b47f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:41.376452 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-b3f4c80f-4657-4b4b-8369-7ee64a16e009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:41.476183 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e79208-6593-4270-8cc3-e0448ad70837 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:41.490996 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa72e0-e936-4f01-8d72-10073b5febb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:41.492967 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa72e0-e936-4f01-8d72-10073b5febb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:41.494710 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa72e0-e936-4f01-8d72-10073b5febb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:41.496019 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa72e0-e936-4f01-8d72-10073b5febb8 admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:41.497585 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfa72e0-e936-4f01-8d72-10073b5febb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:41.502525 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfa72e0-e936-4f01-8d72-10073b5febb8 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:41.547936 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 997/1977] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:41 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:45:41.604965 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f4c80f-4657-4b4b-8369-7ee64a16e009 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:41.606993 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-b3f4c80f-4657-4b4b-8369-7ee64a16e009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:41.635513 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 981/1978] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:45:41 2022] POST /identity/v3/users => generated 300 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:45:41.641876 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa72e0-e936-4f01-8d72-10073b5febb8 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:41.658710 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa72e0-e936-4f01-8d72-10073b5febb8 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:41.667880 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf9659a-fd8e-46d6-a5a7-ed74798b7662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:41.668796 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9659a-fd8e-46d6-a5a7-ed74798b7662 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:41.669268 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9659a-fd8e-46d6-a5a7-ed74798b7662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:41.669639 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9659a-fd8e-46d6-a5a7-ed74798b7662 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:41.678015 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf9659a-fd8e-46d6-a5a7-ed74798b7662 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:41.683341 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf9659a-fd8e-46d6-a5a7-ed74798b7662 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:41.705063 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 982/1979] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:41 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:41.708494 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf9659a-fd8e-46d6-a5a7-ed74798b7662 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:41.727525 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf9659a-fd8e-46d6-a5a7-ed74798b7662 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:41.733516 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f4c80f-4657-4b4b-8369-7ee64a16e009 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:41.736911 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a24df4b-e37b-4bce-bf04-3ebaea53edd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:41.738216 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24df4b-e37b-4bce-bf04-3ebaea53edd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:41.738652 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24df4b-e37b-4bce-bf04-3ebaea53edd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:41.739127 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24df4b-e37b-4bce-bf04-3ebaea53edd4 admin admin] PATH_INFO: `/v3/projects/a12e188c65f64ca2b2afab18bc9a76ed/users/1659b79bf7594e18b84e23f39b0917c8/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:41.745471 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccd3289-c366-42e0-8623-66ed07851649 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:41.746306 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd3289-c366-42e0-8623-66ed07851649 service cinder] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:41.746956 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd3289-c366-42e0-8623-66ed07851649 service cinder] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:41.747534 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd3289-c366-42e0-8623-66ed07851649 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:41.770398 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a24df4b-e37b-4bce-bf04-3ebaea53edd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a12e188c65f64ca2b2afab18bc9a76ed, user_id=1659b79bf7594e18b84e23f39b0917c8, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:41.776510 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a24df4b-e37b-4bce-bf04-3ebaea53edd4 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:41.778398 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccd3289-c366-42e0-8623-66ed07851649 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:41.781282 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccd3289-c366-42e0-8623-66ed07851649 service cinder] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:41.818741 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 998/1980] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:41 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:45:41.831077 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccd3289-c366-42e0-8623-66ed07851649 service cinder] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:41.850512 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccd3289-c366-42e0-8623-66ed07851649 service cinder] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:41.860138 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574252a6-08f9-46bc-acfe-1b59635ef527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:41.861022 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574252a6-08f9-46bc-acfe-1b59635ef527 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:41.861330 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574252a6-08f9-46bc-acfe-1b59635ef527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:41.861662 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574252a6-08f9-46bc-acfe-1b59635ef527 admin admin] PATH_INFO: `/v3/users/de54188935dd430d953ca5c7698b47f2` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:41.864614 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574252a6-08f9-46bc-acfe-1b59635ef527 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de54188935dd430d953ca5c7698b47f2)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:41.867334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574252a6-08f9-46bc-acfe-1b59635ef527 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:41.880149 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 983/1981] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:41 2022] PUT /identity/v3/projects/a12e188c65f64ca2b2afab18bc9a76ed/users/1659b79bf7594e18b84e23f39b0917c8/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:45:41.881488 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a24df4b-e37b-4bce-bf04-3ebaea53edd4 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:41.984024 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a24df4b-e37b-4bce-bf04-3ebaea53edd4 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:41.996191 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380ab51-dad0-43e7-aaf4-812b441a40c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:41.997269 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380ab51-dad0-43e7-aaf4-812b441a40c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:41.998230 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380ab51-dad0-43e7-aaf4-812b441a40c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:41.998883 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380ab51-dad0-43e7-aaf4-812b441a40c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:41.999761 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380ab51-dad0-43e7-aaf4-812b441a40c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:42.006599 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380ab51-dad0-43e7-aaf4-812b441a40c5 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:42.023730 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 984/1982] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:41 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:42.028191 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380ab51-dad0-43e7-aaf4-812b441a40c5 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:42.039530 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380ab51-dad0-43e7-aaf4-812b441a40c5 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:42.047571 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d557a14b-e0be-4656-a2d2-533ac78f222e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:42.048263 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557a14b-e0be-4656-a2d2-533ac78f222e admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:42.049139 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557a14b-e0be-4656-a2d2-533ac78f222e admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:42.049468 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557a14b-e0be-4656-a2d2-533ac78f222e admin admin] PATH_INFO: `/v3/projects/a12e188c65f64ca2b2afab18bc9a76ed/users/1659b79bf7594e18b84e23f39b0917c8/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:42.054037 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d557a14b-e0be-4656-a2d2-533ac78f222e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a12e188c65f64ca2b2afab18bc9a76ed, user_id=1659b79bf7594e18b84e23f39b0917c8, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:42.066535 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d557a14b-e0be-4656-a2d2-533ac78f222e admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:42.088850 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 985/1983] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:42 2022] PUT /identity/v3/projects/a12e188c65f64ca2b2afab18bc9a76ed/users/1659b79bf7594e18b84e23f39b0917c8/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:45:42.096110 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cfd8e2-6519-4c1b-b606-15042377bf0c None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:42.096390 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cfd8e2-6519-4c1b-b606-15042377bf0c None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:42.096805 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cfd8e2-6519-4c1b-b606-15042377bf0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:42.132918 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-16cfd8e2-6519-4c1b-b606-15042377bf0c None None] MFA Rules not processed for user `1659b79bf7594e18b84e23f39b0917c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:42.134287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-16cfd8e2-6519-4c1b-b606-15042377bf0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:42.225691 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 986/1984] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:42 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:45:42.299678 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cfd8e2-6519-4c1b-b606-15042377bf0c None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:42.323103 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-574252a6-08f9-46bc-acfe-1b59635ef527 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de54188935dd430d953ca5c7698b47f2'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:42.385440 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cfd8e2-6519-4c1b-b606-15042377bf0c None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:42.389956 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1152be-8026-4b5e-9b97-7423b5978124 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:42.390609 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1152be-8026-4b5e-9b97-7423b5978124 service cinder] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:42.391041 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1152be-8026-4b5e-9b97-7423b5978124 service cinder] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:42.391322 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1152be-8026-4b5e-9b97-7423b5978124 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:42.402523 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1152be-8026-4b5e-9b97-7423b5978124 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:42.405680 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1152be-8026-4b5e-9b97-7423b5978124 service cinder] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:42.428911 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-574252a6-08f9-46bc-acfe-1b59635ef527 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de54188935dd430d953ca5c7698b47f2'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:42.429149 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 987/1985] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:42 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:45:42.444751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-574252a6-08f9-46bc-acfe-1b59635ef527 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de54188935dd430d953ca5c7698b47f2'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:42.460185 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-574252a6-08f9-46bc-acfe-1b59635ef527 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de54188935dd430d953ca5c7698b47f2'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:42.461245 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-574252a6-08f9-46bc-acfe-1b59635ef527 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de54188935dd430d953ca5c7698b47f2'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:42.475114 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:42.481339 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 999/1986] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:45:41 2022] DELETE /identity/v3/users/de54188935dd430d953ca5c7698b47f2 => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:45:42.481653 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d2163e-570d-4385-bc42-6b578166e437 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:42.481653 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d2163e-570d-4385-bc42-6b578166e437 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:42.481653 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d2163e-570d-4385-bc42-6b578166e437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:42.510138 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-b5d2163e-570d-4385-bc42-6b578166e437 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:42.511704 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-b5d2163e-570d-4385-bc42-6b578166e437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:42.636777 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 988/1987] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:45:42.637238 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:42.652264 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574252a6-08f9-46bc-acfe-1b59635ef527 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:42.655139 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-574252a6-08f9-46bc-acfe-1b59635ef527 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:42.754092 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574252a6-08f9-46bc-acfe-1b59635ef527 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:42.758015 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6f52a9-14f0-4884-a3bf-ffdaa410e721 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:42.758920 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f52a9-14f0-4884-a3bf-ffdaa410e721 service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:42.759316 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f52a9-14f0-4884-a3bf-ffdaa410e721 service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:42.759665 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f52a9-14f0-4884-a3bf-ffdaa410e721 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:42.771797 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6f52a9-14f0-4884-a3bf-ffdaa410e721 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:42.773310 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6f52a9-14f0-4884-a3bf-ffdaa410e721 service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:42.784790 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1000/1988] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:45:43.531710 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d2163e-570d-4385-bc42-6b578166e437 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:43.533281 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-b5d2163e-570d-4385-bc42-6b578166e437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:43.565813 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d2163e-570d-4385-bc42-6b578166e437 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:43.581375 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7a13b-720e-48e7-8a88-e441c21c4233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:43.582327 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7a13b-720e-48e7-8a88-e441c21c4233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:43.582609 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7a13b-720e-48e7-8a88-e441c21c4233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:43.582890 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7a13b-720e-48e7-8a88-e441c21c4233 admin admin] PATH_INFO: `/v3/projects/c1b71be11673464e8f76018ba6ec26c6` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:43.584502 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c7a13b-720e-48e7-8a88-e441c21c4233 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1b71be11673464e8f76018ba6ec26c6)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:43.586438 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c7a13b-720e-48e7-8a88-e441c21c4233 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:43.840278 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-66c7a13b-720e-48e7-8a88-e441c21c4233 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1b71be11673464e8f76018ba6ec26c6', 'request_id': 'req-66c7a13b-720e-48e7-8a88-e441c21c4233'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:43.848269 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-66c7a13b-720e-48e7-8a88-e441c21c4233 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1b71be11673464e8f76018ba6ec26c6', 'request_id': 'req-66c7a13b-720e-48e7-8a88-e441c21c4233'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:43.923619 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-66c7a13b-720e-48e7-8a88-e441c21c4233 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1b71be11673464e8f76018ba6ec26c6', 'request_id': 'req-66c7a13b-720e-48e7-8a88-e441c21c4233'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:43.951901 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-66c7a13b-720e-48e7-8a88-e441c21c4233 admin admin] The token cache is being invalidate because project c1b71be11673464e8f76018ba6ec26c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:45:43.952298 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-66c7a13b-720e-48e7-8a88-e441c21c4233 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:43.953787 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-66c7a13b-720e-48e7-8a88-e441c21c4233 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:43.960853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 989/1989] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:45:43 2022] DELETE /identity/v3/projects/c1b71be11673464e8f76018ba6ec26c6 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:43.963428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7dbcc-5bf8-4406-b786-04b409743718 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:43.963985 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7dbcc-5bf8-4406-b786-04b409743718 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:43.964422 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7dbcc-5bf8-4406-b786-04b409743718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:43.995744 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-88a7dbcc-5bf8-4406-b786-04b409743718 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:43.997099 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-88a7dbcc-5bf8-4406-b786-04b409743718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:44.010895 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:44.077068 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:44.084220 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1001/1990] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:44.092219 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c65e68-1529-456a-8efb-0951f789c76b None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:44.092864 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c65e68-1529-456a-8efb-0951f789c76b None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:44.093192 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c65e68-1529-456a-8efb-0951f789c76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:44.115821 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-09c65e68-1529-456a-8efb-0951f789c76b None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:44.117100 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-09c65e68-1529-456a-8efb-0951f789c76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:44.125600 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 990/1991] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:44.130094 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a7dbcc-5bf8-4406-b786-04b409743718 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:44.148059 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a7dbcc-5bf8-4406-b786-04b409743718 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:44.153709 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3394b6bd-4253-4c05-9e56-3bf9eaed2bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:44.154470 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394b6bd-4253-4c05-9e56-3bf9eaed2bd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:44.154850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394b6bd-4253-4c05-9e56-3bf9eaed2bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:44.155192 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394b6bd-4253-4c05-9e56-3bf9eaed2bd2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:44.155932 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3394b6bd-4253-4c05-9e56-3bf9eaed2bd2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:45:44.156301 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3394b6bd-4253-4c05-9e56-3bf9eaed2bd2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:44.158052 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3394b6bd-4253-4c05-9e56-3bf9eaed2bd2 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:44.171504 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1002/1992] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:45:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:45:44.176256 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c65e68-1529-456a-8efb-0951f789c76b None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:44.190823 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c65e68-1529-456a-8efb-0951f789c76b None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:44.195288 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c70d14-33f0-4c9a-9989-53e54934fbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:44.195907 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c70d14-33f0-4c9a-9989-53e54934fbdb admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:44.196183 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c70d14-33f0-4c9a-9989-53e54934fbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:44.196510 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c70d14-33f0-4c9a-9989-53e54934fbdb admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:44.197145 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c70d14-33f0-4c9a-9989-53e54934fbdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:44.199141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c70d14-33f0-4c9a-9989-53e54934fbdb admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:44.392443 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 991/1993] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:45:44 2022] POST /identity/v3/projects => generated 367 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:44.396790 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3394b6bd-4253-4c05-9e56-3bf9eaed2bd2 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:44.411553 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3394b6bd-4253-4c05-9e56-3bf9eaed2bd2 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:44.484219 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b9ecf-9151-4844-917c-fca6e8eb3c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:44.484957 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b9ecf-9151-4844-917c-fca6e8eb3c9c admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:44.485242 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b9ecf-9151-4844-917c-fca6e8eb3c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:44.485520 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b9ecf-9151-4844-917c-fca6e8eb3c9c admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:44.486156 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56b9ecf-9151-4844-917c-fca6e8eb3c9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:44.488154 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56b9ecf-9151-4844-917c-fca6e8eb3c9c admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:44.645935 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1003/1994] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:45:44 2022] POST /identity/v3/users => generated 284 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:44.647275 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c70d14-33f0-4c9a-9989-53e54934fbdb admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:44.658074 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c70d14-33f0-4c9a-9989-53e54934fbdb admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:44.663577 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e977de8-303f-44f8-831e-d99a9858fea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:44.665140 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e977de8-303f-44f8-831e-d99a9858fea4 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:44.665419 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e977de8-303f-44f8-831e-d99a9858fea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:44.665691 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e977de8-303f-44f8-831e-d99a9858fea4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:44.666610 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e977de8-303f-44f8-831e-d99a9858fea4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:44.668225 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e977de8-303f-44f8-831e-d99a9858fea4 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:44.679205 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 992/1995] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:44.684059 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b9ecf-9151-4844-917c-fca6e8eb3c9c admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:44.697553 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:44.697805 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b9ecf-9151-4844-917c-fca6e8eb3c9c admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:44.703700 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8c1c08-e924-43e5-8b6f-0ef620ece040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:44.704521 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8c1c08-e924-43e5-8b6f-0ef620ece040 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:44.704812 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8c1c08-e924-43e5-8b6f-0ef620ece040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:44.705094 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8c1c08-e924-43e5-8b6f-0ef620ece040 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:44.705769 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8c1c08-e924-43e5-8b6f-0ef620ece040 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:44.712837 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8c1c08-e924-43e5-8b6f-0ef620ece040 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:44.719278 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1004/1996] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:44.724264 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e977de8-303f-44f8-831e-d99a9858fea4 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:44.736337 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e977de8-303f-44f8-831e-d99a9858fea4 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:44.750249 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ea8f9-f0f4-4c0e-9784-30159cce804a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:44.750249 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ea8f9-f0f4-4c0e-9784-30159cce804a admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:44.750249 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ea8f9-f0f4-4c0e-9784-30159cce804a admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:44.750249 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ea8f9-f0f4-4c0e-9784-30159cce804a admin admin] PATH_INFO: `/v3/projects/1578db0219964206875af31847275d90/users/e44a0edca1604fb2aee88420a77ab822/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:44.766253 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699ea8f9-f0f4-4c0e-9784-30159cce804a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1578db0219964206875af31847275d90, user_id=e44a0edca1604fb2aee88420a77ab822, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:44.770328 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699ea8f9-f0f4-4c0e-9784-30159cce804a admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:44.869390 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 993/1997] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:44 2022] PUT /identity/v3/projects/1578db0219964206875af31847275d90/users/e44a0edca1604fb2aee88420a77ab822/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:44.880634 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1164f87a-c035-4ca0-803f-6a18db90cd29 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:44.880921 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1164f87a-c035-4ca0-803f-6a18db90cd29 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:44.881199 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1164f87a-c035-4ca0-803f-6a18db90cd29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:44.911338 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-1164f87a-c035-4ca0-803f-6a18db90cd29 None None] MFA Rules not processed for user `e44a0edca1604fb2aee88420a77ab822`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:44.912632 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-1164f87a-c035-4ca0-803f-6a18db90cd29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:45.009611 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1005/1998] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:45.037623 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ea8f9-f0f4-4c0e-9784-30159cce804a admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:45.040160 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-699ea8f9-f0f4-4c0e-9784-30159cce804a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:45.041951 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1164f87a-c035-4ca0-803f-6a18db90cd29 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:45.044039 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-1164f87a-c035-4ca0-803f-6a18db90cd29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:45.175389 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ea8f9-f0f4-4c0e-9784-30159cce804a admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:45.191040 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8301cc6-9be0-4988-bc17-57291babd33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:45.193423 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8301cc6-9be0-4988-bc17-57291babd33f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:45.194408 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8301cc6-9be0-4988-bc17-57291babd33f admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:45.195614 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8301cc6-9be0-4988-bc17-57291babd33f admin admin] PATH_INFO: `/v3/users/1cc9623a9d614091b3716fca1bf4a57d` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:45.198432 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8301cc6-9be0-4988-bc17-57291babd33f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cc9623a9d614091b3716fca1bf4a57d)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:45.202776 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8301cc6-9be0-4988-bc17-57291babd33f admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:45.218364 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:45.285918 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1164f87a-c035-4ca0-803f-6a18db90cd29 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:45.297920 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12cf39-0d8b-4526-b7b3-da2e9dede231 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9aee9c524a974a248c3bf363444f584a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:45.298816 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12cf39-0d8b-4526-b7b3-da2e9dede231 service glance] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:45.299105 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12cf39-0d8b-4526-b7b3-da2e9dede231 service glance] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:45.299380 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12cf39-0d8b-4526-b7b3-da2e9dede231 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:45.326181 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e12cf39-0d8b-4526-b7b3-da2e9dede231 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:45.327841 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e12cf39-0d8b-4526-b7b3-da2e9dede231 service glance] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:45.352349 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1006/1999] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:45:45.354225 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12cf39-0d8b-4526-b7b3-da2e9dede231 service glance] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:45.355905 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-7e12cf39-0d8b-4526-b7b3-da2e9dede231 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:45.436834 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12cf39-0d8b-4526-b7b3-da2e9dede231 service glance] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:45.442080 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742262e-a5c3-45fe-8f4a-ddee19f1d918 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:45.442717 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742262e-a5c3-45fe-8f4a-ddee19f1d918 service cinder] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:45.443010 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742262e-a5c3-45fe-8f4a-ddee19f1d918 service cinder] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:45.443286 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742262e-a5c3-45fe-8f4a-ddee19f1d918 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:45.445394 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-1742262e-a5c3-45fe-8f4a-ddee19f1d918 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:45.473326 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:45.500563 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-c8301cc6-9be0-4988-bc17-57291babd33f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cc9623a9d614091b3716fca1bf4a57d'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:45.515841 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-c8301cc6-9be0-4988-bc17-57291babd33f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cc9623a9d614091b3716fca1bf4a57d'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:45.516723 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-c8301cc6-9be0-4988-bc17-57291babd33f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cc9623a9d614091b3716fca1bf4a57d'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:45.541219 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1742262e-a5c3-45fe-8f4a-ddee19f1d918 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:45.542769 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1742262e-a5c3-45fe-8f4a-ddee19f1d918 service cinder] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:45.553015 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1007/2000] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:45:45.707052 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-c8301cc6-9be0-4988-bc17-57291babd33f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cc9623a9d614091b3716fca1bf4a57d'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:45.724854 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-c8301cc6-9be0-4988-bc17-57291babd33f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cc9623a9d614091b3716fca1bf4a57d'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:45.731737 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 994/2001] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:45:45 2022] DELETE /identity/v3/users/1cc9623a9d614091b3716fca1bf4a57d => generated 0 bytes in 695 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:45.736572 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514e5f0-58d9-4541-8a92-9366a9479dd5 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:45.736878 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514e5f0-58d9-4541-8a92-9366a9479dd5 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:45.737700 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514e5f0-58d9-4541-8a92-9366a9479dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:45.760741 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-f514e5f0-58d9-4541-8a92-9366a9479dd5 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:45.762037 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-f514e5f0-58d9-4541-8a92-9366a9479dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:45.830294 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1008/2002] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:45.840342 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8301cc6-9be0-4988-bc17-57291babd33f admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:45.841789 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-c8301cc6-9be0-4988-bc17-57291babd33f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:45.915783 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8301cc6-9be0-4988-bc17-57291babd33f admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:45.919574 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98467905-ab8b-4158-b590-f124e7e6b902 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:45.920177 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98467905-ab8b-4158-b590-f124e7e6b902 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:45.920495 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98467905-ab8b-4158-b590-f124e7e6b902 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:45.920773 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98467905-ab8b-4158-b590-f124e7e6b902 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:45.930947 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98467905-ab8b-4158-b590-f124e7e6b902 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:45.935248 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98467905-ab8b-4158-b590-f124e7e6b902 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:45.942672 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 995/2003] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:45:46.170772 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:46.317054 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:46.427455 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f514e5f0-58d9-4541-8a92-9366a9479dd5 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:46.428959 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-f514e5f0-58d9-4541-8a92-9366a9479dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:46.454733 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f514e5f0-58d9-4541-8a92-9366a9479dd5 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:46.470490 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5a4f5-56e9-488a-8b0e-e42b9438ddb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:46.471168 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5a4f5-56e9-488a-8b0e-e42b9438ddb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:46.474641 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5a4f5-56e9-488a-8b0e-e42b9438ddb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:46.478450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5a4f5-56e9-488a-8b0e-e42b9438ddb5 admin admin] PATH_INFO: `/v3/projects/7fb3224a7adc4264b0a1e585d39a0c14` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:46.478639 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d5a4f5-56e9-488a-8b0e-e42b9438ddb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fb3224a7adc4264b0a1e585d39a0c14)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:46.493306 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d5a4f5-56e9-488a-8b0e-e42b9438ddb5 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:46.702917 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-50d5a4f5-56e9-488a-8b0e-e42b9438ddb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fb3224a7adc4264b0a1e585d39a0c14', 'request_id': 'req-50d5a4f5-56e9-488a-8b0e-e42b9438ddb5'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:46.712612 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-50d5a4f5-56e9-488a-8b0e-e42b9438ddb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fb3224a7adc4264b0a1e585d39a0c14', 'request_id': 'req-50d5a4f5-56e9-488a-8b0e-e42b9438ddb5'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:46.840468 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-50d5a4f5-56e9-488a-8b0e-e42b9438ddb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fb3224a7adc4264b0a1e585d39a0c14', 'request_id': 'req-50d5a4f5-56e9-488a-8b0e-e42b9438ddb5'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:46.855881 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-50d5a4f5-56e9-488a-8b0e-e42b9438ddb5 admin admin] The token cache is being invalidate because project 7fb3224a7adc4264b0a1e585d39a0c14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:45:46.856174 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-50d5a4f5-56e9-488a-8b0e-e42b9438ddb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:46.856955 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-50d5a4f5-56e9-488a-8b0e-e42b9438ddb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:46.858555 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1009/2004] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:45:46 2022] DELETE /identity/v3/projects/7fb3224a7adc4264b0a1e585d39a0c14 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:46.862994 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98467905-ab8b-4158-b590-f124e7e6b902 service neutron] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:46.873604 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98467905-ab8b-4158-b590-f124e7e6b902 service neutron] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:46.959728 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bba0ca-5777-40b7-b935-01703184233b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:46.960365 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bba0ca-5777-40b7-b935-01703184233b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:46.960727 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bba0ca-5777-40b7-b935-01703184233b admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:46.961056 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bba0ca-5777-40b7-b935-01703184233b admin admin] PATH_INFO: `/v3/users/bab41be5f8944d55b4cc3ba73c9fee24` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:46.962189 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bba0ca-5777-40b7-b935-01703184233b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bab41be5f8944d55b4cc3ba73c9fee24)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:46.964371 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bba0ca-5777-40b7-b935-01703184233b admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:47.229491 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-12bba0ca-5777-40b7-b935-01703184233b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bab41be5f8944d55b4cc3ba73c9fee24'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:47.242382 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-12bba0ca-5777-40b7-b935-01703184233b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bab41be5f8944d55b4cc3ba73c9fee24'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:47.243208 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-12bba0ca-5777-40b7-b935-01703184233b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bab41be5f8944d55b4cc3ba73c9fee24'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:47.363917 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-12bba0ca-5777-40b7-b935-01703184233b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bab41be5f8944d55b4cc3ba73c9fee24'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:47.382250 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-12bba0ca-5777-40b7-b935-01703184233b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bab41be5f8944d55b4cc3ba73c9fee24'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:47.390542 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 996/2005] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:45:46 2022] DELETE /identity/v3/users/bab41be5f8944d55b4cc3ba73c9fee24 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:45:47.566648 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:48.149058 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5a4f5-56e9-488a-8b0e-e42b9438ddb5 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:48.153036 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-50d5a4f5-56e9-488a-8b0e-e42b9438ddb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:48.202950 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5a4f5-56e9-488a-8b0e-e42b9438ddb5 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:48.229321 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:48.279987 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72379128-f041-4743-bb12-b680a68841c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:48.280613 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72379128-f041-4743-bb12-b680a68841c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:48.280900 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72379128-f041-4743-bb12-b680a68841c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:48.281173 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72379128-f041-4743-bb12-b680a68841c4 admin admin] PATH_INFO: `/v3/projects/f2122143ce924f7680452df87cad747d` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:48.282430 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72379128-f041-4743-bb12-b680a68841c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2122143ce924f7680452df87cad747d)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:48.284762 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72379128-f041-4743-bb12-b680a68841c4 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:48.487638 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-72379128-f041-4743-bb12-b680a68841c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2122143ce924f7680452df87cad747d', 'request_id': 'req-72379128-f041-4743-bb12-b680a68841c4'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:48.496235 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-72379128-f041-4743-bb12-b680a68841c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2122143ce924f7680452df87cad747d', 'request_id': 'req-72379128-f041-4743-bb12-b680a68841c4'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:48.587417 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-72379128-f041-4743-bb12-b680a68841c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2122143ce924f7680452df87cad747d', 'request_id': 'req-72379128-f041-4743-bb12-b680a68841c4'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:48.600754 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-72379128-f041-4743-bb12-b680a68841c4 admin admin] The token cache is being invalidate because project f2122143ce924f7680452df87cad747d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:45:48.601099 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-72379128-f041-4743-bb12-b680a68841c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:48.601786 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-72379128-f041-4743-bb12-b680a68841c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:48.603439 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1010/2006] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:45:48 2022] DELETE /identity/v3/projects/f2122143ce924f7680452df87cad747d => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:48.608903 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678dbed-090d-43ef-b2f7-fc66f3e0c909 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:48.609215 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678dbed-090d-43ef-b2f7-fc66f3e0c909 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:48.609528 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678dbed-090d-43ef-b2f7-fc66f3e0c909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:48.639137 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-f678dbed-090d-43ef-b2f7-fc66f3e0c909 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:48.641276 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-f678dbed-090d-43ef-b2f7-fc66f3e0c909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:48.748598 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 997/2007] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:45:48.759890 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af044534-abaf-433f-968e-5f9bfa70b740 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:48.760505 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af044534-abaf-433f-968e-5f9bfa70b740 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:48.761172 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af044534-abaf-433f-968e-5f9bfa70b740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:48.786842 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-af044534-abaf-433f-968e-5f9bfa70b740 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:48.788213 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-af044534-abaf-433f-968e-5f9bfa70b740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:48.803133 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1011/2008] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:48.808991 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f678dbed-090d-43ef-b2f7-fc66f3e0c909 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:48.824581 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f678dbed-090d-43ef-b2f7-fc66f3e0c909 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:48.834077 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e467fc-6a9a-49a7-9736-eed793a02156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:48.834732 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e467fc-6a9a-49a7-9736-eed793a02156 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:48.835016 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e467fc-6a9a-49a7-9736-eed793a02156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:48.835324 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e467fc-6a9a-49a7-9736-eed793a02156 admin admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:48.836185 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e467fc-6a9a-49a7-9736-eed793a02156 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:45:48.836573 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e467fc-6a9a-49a7-9736-eed793a02156 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:48.838491 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e467fc-6a9a-49a7-9736-eed793a02156 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:48.852957 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 998/2009] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:45:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:45:48.858695 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af044534-abaf-433f-968e-5f9bfa70b740 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:48.870434 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af044534-abaf-433f-968e-5f9bfa70b740 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:48.875582 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6688813-de01-43a8-8c5d-bc182c1ccd02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:48.876272 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6688813-de01-43a8-8c5d-bc182c1ccd02 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:48.876623 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6688813-de01-43a8-8c5d-bc182c1ccd02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:48.876900 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6688813-de01-43a8-8c5d-bc182c1ccd02 admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:48.877699 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6688813-de01-43a8-8c5d-bc182c1ccd02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:48.879690 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6688813-de01-43a8-8c5d-bc182c1ccd02 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:48.903022 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:48.995118 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1012/2010] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:45:48 2022] POST /identity/v3/projects => generated 381 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:45:49.000194 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e467fc-6a9a-49a7-9736-eed793a02156 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:49.014396 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e467fc-6a9a-49a7-9736-eed793a02156 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:49.050272 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6688813-de01-43a8-8c5d-bc182c1ccd02 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:49.053725 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-f6688813-de01-43a8-8c5d-bc182c1ccd02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:49.100658 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e7054-a601-4f18-ad7f-8c0d4af41043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:49.101299 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e7054-a601-4f18-ad7f-8c0d4af41043 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:49.101644 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e7054-a601-4f18-ad7f-8c0d4af41043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:49.101933 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e7054-a601-4f18-ad7f-8c0d4af41043 admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:49.102771 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e7054-a601-4f18-ad7f-8c0d4af41043 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:49.105188 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e7054-a601-4f18-ad7f-8c0d4af41043 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:49.156349 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6688813-de01-43a8-8c5d-bc182c1ccd02 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:49.163685 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075b2c0d-6d1a-4a0e-946d-11a669b5eb1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:49.164504 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b2c0d-6d1a-4a0e-946d-11a669b5eb1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:49.164876 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b2c0d-6d1a-4a0e-946d-11a669b5eb1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:49.165362 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b2c0d-6d1a-4a0e-946d-11a669b5eb1c admin admin] PATH_INFO: `/v3/users/5dc68e2ba7d44470b97b7a80a1808af5` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:49.169460 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075b2c0d-6d1a-4a0e-946d-11a669b5eb1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dc68e2ba7d44470b97b7a80a1808af5)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:49.169460 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075b2c0d-6d1a-4a0e-946d-11a669b5eb1c admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:49.227446 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 999/2011] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:45:48 2022] POST /identity/v3/users => generated 291 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:45:49.230924 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e7054-a601-4f18-ad7f-8c0d4af41043 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:49.257259 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e7054-a601-4f18-ad7f-8c0d4af41043 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:49.267393 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304dc0e9-6b9e-42b1-a174-481ae28a97cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:49.270052 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304dc0e9-6b9e-42b1-a174-481ae28a97cd admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:49.271070 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304dc0e9-6b9e-42b1-a174-481ae28a97cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:49.271474 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304dc0e9-6b9e-42b1-a174-481ae28a97cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:49.272511 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304dc0e9-6b9e-42b1-a174-481ae28a97cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:49.275559 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304dc0e9-6b9e-42b1-a174-481ae28a97cd admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:49.305386 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1000/2012] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:49 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:49.311524 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304dc0e9-6b9e-42b1-a174-481ae28a97cd admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:49.324181 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304dc0e9-6b9e-42b1-a174-481ae28a97cd admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:49.329818 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b776d8a3-bfef-4597-aac6-57730c97e56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:49.332595 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776d8a3-bfef-4597-aac6-57730c97e56a admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:49.332986 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776d8a3-bfef-4597-aac6-57730c97e56a admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:49.333522 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776d8a3-bfef-4597-aac6-57730c97e56a admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:49.334643 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b776d8a3-bfef-4597-aac6-57730c97e56a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:49.336614 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b776d8a3-bfef-4597-aac6-57730c97e56a admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:49.360122 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1001/2013] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:49 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:45:49.360122 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b776d8a3-bfef-4597-aac6-57730c97e56a admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:49.369801 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b776d8a3-bfef-4597-aac6-57730c97e56a admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:49.375543 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519a4340-eba1-4379-8bce-49f0ac2ce52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:49.380372 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a4340-eba1-4379-8bce-49f0ac2ce52b admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:49.380372 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a4340-eba1-4379-8bce-49f0ac2ce52b admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:49.380372 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a4340-eba1-4379-8bce-49f0ac2ce52b admin admin] PATH_INFO: `/v3/projects/233b2e73d7824d09b0539f80753731c0/users/d42ed50bb9fe4701b85a413b9e23d888/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:49.398530 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519a4340-eba1-4379-8bce-49f0ac2ce52b admin admin] RBAC: Authorizing `identity:create_grant(project_id=233b2e73d7824d09b0539f80753731c0, user_id=d42ed50bb9fe4701b85a413b9e23d888, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:49.401317 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519a4340-eba1-4379-8bce-49f0ac2ce52b admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:49.446225 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-075b2c0d-6d1a-4a0e-946d-11a669b5eb1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dc68e2ba7d44470b97b7a80a1808af5'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:49.528501 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1002/2014] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:49 2022] PUT /identity/v3/projects/233b2e73d7824d09b0539f80753731c0/users/d42ed50bb9fe4701b85a413b9e23d888/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:45:49.547747 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27caf17b-9d7c-47e9-981d-c4f379d24947 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:49.547981 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27caf17b-9d7c-47e9-981d-c4f379d24947 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:49.548975 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27caf17b-9d7c-47e9-981d-c4f379d24947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:49.554824 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-075b2c0d-6d1a-4a0e-946d-11a669b5eb1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dc68e2ba7d44470b97b7a80a1808af5'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:49.566834 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-075b2c0d-6d1a-4a0e-946d-11a669b5eb1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dc68e2ba7d44470b97b7a80a1808af5'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:49.567138 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:49.583463 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-075b2c0d-6d1a-4a0e-946d-11a669b5eb1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dc68e2ba7d44470b97b7a80a1808af5'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:49.596101 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-075b2c0d-6d1a-4a0e-946d-11a669b5eb1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dc68e2ba7d44470b97b7a80a1808af5'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:49.602512 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-27caf17b-9d7c-47e9-981d-c4f379d24947 None None] MFA Rules not processed for user `d42ed50bb9fe4701b85a413b9e23d888`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:49.603973 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-27caf17b-9d7c-47e9-981d-c4f379d24947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:49.607743 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1013/2015] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:45:49 2022] DELETE /identity/v3/users/5dc68e2ba7d44470b97b7a80a1808af5 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:45:49.613591 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46dd56d-bf4c-44a3-bef9-ca9c63a3ab55 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:49.614079 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46dd56d-bf4c-44a3-bef9-ca9c63a3ab55 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:49.614387 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46dd56d-bf4c-44a3-bef9-ca9c63a3ab55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:49.637919 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-c46dd56d-bf4c-44a3-bef9-ca9c63a3ab55 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:49.639471 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-c46dd56d-bf4c-44a3-bef9-ca9c63a3ab55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:49.710260 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1003/2016] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:45:49.715364 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27caf17b-9d7c-47e9-981d-c4f379d24947 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:49.716816 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-27caf17b-9d7c-47e9-981d-c4f379d24947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:49.720617 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1014/2017] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:45:49.729113 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46dd56d-bf4c-44a3-bef9-ca9c63a3ab55 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:49.730581 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-c46dd56d-bf4c-44a3-bef9-ca9c63a3ab55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:49.734272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27caf17b-9d7c-47e9-981d-c4f379d24947 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:49.739654 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d976358-3d0b-4acb-a33b-e00c85d011c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:49.740283 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d976358-3d0b-4acb-a33b-e00c85d011c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:49.740607 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d976358-3d0b-4acb-a33b-e00c85d011c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:49.740891 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d976358-3d0b-4acb-a33b-e00c85d011c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:49.741559 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d976358-3d0b-4acb-a33b-e00c85d011c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:49.743603 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d976358-3d0b-4acb-a33b-e00c85d011c7 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:49.807678 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46dd56d-bf4c-44a3-bef9-ca9c63a3ab55 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:49.814360 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228d0b11-d13f-4a38-8759-e7dbf508a489 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:49.816125 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d0b11-d13f-4a38-8759-e7dbf508a489 service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:49.816638 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d0b11-d13f-4a38-8759-e7dbf508a489 service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:49.817173 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d0b11-d13f-4a38-8759-e7dbf508a489 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:49.833549 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228d0b11-d13f-4a38-8759-e7dbf508a489 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:49.836245 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228d0b11-d13f-4a38-8759-e7dbf508a489 service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:49.842093 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1004/2018] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:45:49 2022] POST /identity/v3/projects => generated 381 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:45:49.846405 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d976358-3d0b-4acb-a33b-e00c85d011c7 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:49.849414 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1015/2019] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:45:49.945931 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d976358-3d0b-4acb-a33b-e00c85d011c7 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:49.954568 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f344f-6cb6-48b3-9377-251fd5177f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:49.956254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f344f-6cb6-48b3-9377-251fd5177f2a admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:49.957537 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f344f-6cb6-48b3-9377-251fd5177f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:49.958158 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f344f-6cb6-48b3-9377-251fd5177f2a admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:49.960149 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9f344f-6cb6-48b3-9377-251fd5177f2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:49.962874 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9f344f-6cb6-48b3-9377-251fd5177f2a admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:49.990163 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:50.094755 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1005/2020] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:45:49 2022] POST /identity/v3/users => generated 297 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:50.096276 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228d0b11-d13f-4a38-8759-e7dbf508a489 service neutron] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:50.119291 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228d0b11-d13f-4a38-8759-e7dbf508a489 service neutron] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:50.124325 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fffc0c0-9777-44c7-93cc-fc1cf9291077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:50.124975 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffc0c0-9777-44c7-93cc-fc1cf9291077 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:50.125255 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffc0c0-9777-44c7-93cc-fc1cf9291077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:50.125560 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffc0c0-9777-44c7-93cc-fc1cf9291077 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:50.126769 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fffc0c0-9777-44c7-93cc-fc1cf9291077 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:50.128796 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fffc0c0-9777-44c7-93cc-fc1cf9291077 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:50.140515 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1016/2021] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:50 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:50.145382 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f344f-6cb6-48b3-9377-251fd5177f2a admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:50.156482 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f344f-6cb6-48b3-9377-251fd5177f2a admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:50.162401 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673091d1-b260-4ebf-ba64-1e15c144074d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:50.163122 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673091d1-b260-4ebf-ba64-1e15c144074d admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:50.163407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673091d1-b260-4ebf-ba64-1e15c144074d admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:50.163818 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673091d1-b260-4ebf-ba64-1e15c144074d admin admin] PATH_INFO: `/v3/projects/0e9c17042b04463dbf6aae91773e209b/users/f99751fff8024584998dbc0325d27959/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:50.186674 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673091d1-b260-4ebf-ba64-1e15c144074d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e9c17042b04463dbf6aae91773e209b, user_id=f99751fff8024584998dbc0325d27959, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:50.189401 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673091d1-b260-4ebf-ba64-1e15c144074d admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:50.207167 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:50.269992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1006/2022] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:50 2022] PUT /identity/v3/projects/0e9c17042b04463dbf6aae91773e209b/users/f99751fff8024584998dbc0325d27959/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:50.274199 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fffc0c0-9777-44c7-93cc-fc1cf9291077 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:50.285859 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fffc0c0-9777-44c7-93cc-fc1cf9291077 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:50.293669 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:50.372987 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673091d1-b260-4ebf-ba64-1e15c144074d admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:50.374171 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aa4d58-4ed1-42f5-bf51-fc0bfbd70c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:50.374810 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa4d58-4ed1-42f5-bf51-fc0bfbd70c2a admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:50.375159 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa4d58-4ed1-42f5-bf51-fc0bfbd70c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:50.375435 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa4d58-4ed1-42f5-bf51-fc0bfbd70c2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:50.376433 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aa4d58-4ed1-42f5-bf51-fc0bfbd70c2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:50.376769 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-673091d1-b260-4ebf-ba64-1e15c144074d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:50.379290 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aa4d58-4ed1-42f5-bf51-fc0bfbd70c2a admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:50.399201 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1017/2023] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:50 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:50.409918 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673091d1-b260-4ebf-ba64-1e15c144074d admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:50.410552 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aa4d58-4ed1-42f5-bf51-fc0bfbd70c2a admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:50.421895 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2692b979-b338-4b5f-9d05-a43cc8244fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:50.423925 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692b979-b338-4b5f-9d05-a43cc8244fd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:50.425771 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692b979-b338-4b5f-9d05-a43cc8244fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:50.425956 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aa4d58-4ed1-42f5-bf51-fc0bfbd70c2a admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:50.426748 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692b979-b338-4b5f-9d05-a43cc8244fd5 admin admin] PATH_INFO: `/v3/projects/6e8667d341434925adf4096426e5b55b` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:50.429727 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2692b979-b338-4b5f-9d05-a43cc8244fd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e8667d341434925adf4096426e5b55b)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:50.431917 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bcaec6-6a30-4c02-a4d4-af814e276dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:50.432862 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bcaec6-6a30-4c02-a4d4-af814e276dfb admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:50.433252 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bcaec6-6a30-4c02-a4d4-af814e276dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:50.433612 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bcaec6-6a30-4c02-a4d4-af814e276dfb admin admin] PATH_INFO: `/v3/projects/0e9c17042b04463dbf6aae91773e209b/users/f99751fff8024584998dbc0325d27959/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:50.433834 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2692b979-b338-4b5f-9d05-a43cc8244fd5 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:50.436675 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bcaec6-6a30-4c02-a4d4-af814e276dfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e9c17042b04463dbf6aae91773e209b, user_id=f99751fff8024584998dbc0325d27959, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:50.440697 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bcaec6-6a30-4c02-a4d4-af814e276dfb admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:50.460281 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1018/2024] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:50 2022] PUT /identity/v3/projects/0e9c17042b04463dbf6aae91773e209b/users/f99751fff8024584998dbc0325d27959/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:50.464906 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:50.472479 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827e264-5a42-4229-b54f-cef14b02eefb None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:50.473104 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827e264-5a42-4229-b54f-cef14b02eefb None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:50.473573 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827e264-5a42-4229-b54f-cef14b02eefb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:50.519702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-1827e264-5a42-4229-b54f-cef14b02eefb None None] MFA Rules not processed for user `f99751fff8024584998dbc0325d27959`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:50.521306 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-1827e264-5a42-4229-b54f-cef14b02eefb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:50.574688 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-2692b979-b338-4b5f-9d05-a43cc8244fd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e8667d341434925adf4096426e5b55b', 'request_id': 'req-2692b979-b338-4b5f-9d05-a43cc8244fd5'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:50.592741 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-2692b979-b338-4b5f-9d05-a43cc8244fd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e8667d341434925adf4096426e5b55b', 'request_id': 'req-2692b979-b338-4b5f-9d05-a43cc8244fd5'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:50.627247 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1019/2025] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:50 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:50.699488 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827e264-5a42-4229-b54f-cef14b02eefb None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:50.702016 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-1827e264-5a42-4229-b54f-cef14b02eefb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:50.723827 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-2692b979-b338-4b5f-9d05-a43cc8244fd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e8667d341434925adf4096426e5b55b', 'request_id': 'req-2692b979-b338-4b5f-9d05-a43cc8244fd5'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:50.739502 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-2692b979-b338-4b5f-9d05-a43cc8244fd5 admin admin] The token cache is being invalidate because project 6e8667d341434925adf4096426e5b55b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:45:50.740060 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-2692b979-b338-4b5f-9d05-a43cc8244fd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:50.741241 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-2692b979-b338-4b5f-9d05-a43cc8244fd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:50.743890 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1007/2026] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:45:50 2022] DELETE /identity/v3/projects/6e8667d341434925adf4096426e5b55b => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:50.748198 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2692b979-b338-4b5f-9d05-a43cc8244fd5 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:50.749800 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-2692b979-b338-4b5f-9d05-a43cc8244fd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:50.795792 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827e264-5a42-4229-b54f-cef14b02eefb None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:50.852702 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2692b979-b338-4b5f-9d05-a43cc8244fd5 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:50.860592 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9c4c53-ccf7-46f5-b259-c79098aee8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:50.861718 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c4c53-ccf7-46f5-b259-c79098aee8ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:50.861718 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c4c53-ccf7-46f5-b259-c79098aee8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:50.861718 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c4c53-ccf7-46f5-b259-c79098aee8ae admin admin] PATH_INFO: `/v3/users/74c44a550c54451aae6bc751b57c7b4a` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:50.865183 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9c4c53-ccf7-46f5-b259-c79098aee8ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=74c44a550c54451aae6bc751b57c7b4a)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:50.868932 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9c4c53-ccf7-46f5-b259-c79098aee8ae admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:50.895049 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ab7497-f7d5-42fb-8411-bb87c6487191 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:50.896201 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab7497-f7d5-42fb-8411-bb87c6487191 service cinder] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:50.896686 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab7497-f7d5-42fb-8411-bb87c6487191 service cinder] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:50.897001 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab7497-f7d5-42fb-8411-bb87c6487191 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:50.897971 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:50.899661 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-03ab7497-f7d5-42fb-8411-bb87c6487191 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:50.985994 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ab7497-f7d5-42fb-8411-bb87c6487191 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:50.987695 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ab7497-f7d5-42fb-8411-bb87c6487191 service cinder] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:50.998808 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1020/2027] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:50 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:45:51.204040 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-ea9c4c53-ccf7-46f5-b259-c79098aee8ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74c44a550c54451aae6bc751b57c7b4a'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:51.217710 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-ea9c4c53-ccf7-46f5-b259-c79098aee8ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74c44a550c54451aae6bc751b57c7b4a'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:51.218559 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-ea9c4c53-ccf7-46f5-b259-c79098aee8ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74c44a550c54451aae6bc751b57c7b4a'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:51.286608 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ab7497-f7d5-42fb-8411-bb87c6487191 service cinder] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:51.288803 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-03ab7497-f7d5-42fb-8411-bb87c6487191 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:51.300809 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ab7497-f7d5-42fb-8411-bb87c6487191 service cinder] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:51.310494 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca4ae1-5fa4-4def-9933-0d48fc2e6465 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:51.311133 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca4ae1-5fa4-4def-9933-0d48fc2e6465 service cinder] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:51.311415 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca4ae1-5fa4-4def-9933-0d48fc2e6465 service cinder] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:51.311696 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca4ae1-5fa4-4def-9933-0d48fc2e6465 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:51.313750 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-6bca4ae1-5fa4-4def-9933-0d48fc2e6465 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:51.354692 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-ea9c4c53-ccf7-46f5-b259-c79098aee8ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74c44a550c54451aae6bc751b57c7b4a'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:51.355926 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:51.369838 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-ea9c4c53-ccf7-46f5-b259-c79098aee8ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74c44a550c54451aae6bc751b57c7b4a'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:51.376671 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1008/2028] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:45:50 2022] DELETE /identity/v3/users/74c44a550c54451aae6bc751b57c7b4a => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:45:51.399399 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bca4ae1-5fa4-4def-9933-0d48fc2e6465 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:51.400940 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bca4ae1-5fa4-4def-9933-0d48fc2e6465 service cinder] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:51.414974 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1021/2029] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:51 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:45:52.035384 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9c4c53-ccf7-46f5-b259-c79098aee8ae admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:52.037896 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-ea9c4c53-ccf7-46f5-b259-c79098aee8ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:52.135648 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9c4c53-ccf7-46f5-b259-c79098aee8ae admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:52.141007 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466f156f-a1ee-4b01-812a-0a7e37447a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:52.142062 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f156f-a1ee-4b01-812a-0a7e37447a47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:52.142546 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f156f-a1ee-4b01-812a-0a7e37447a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:52.143109 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f156f-a1ee-4b01-812a-0a7e37447a47 admin admin] PATH_INFO: `/v3/projects/0074b22dffe2425f90dbd82c0ed91b67` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:52.145396 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466f156f-a1ee-4b01-812a-0a7e37447a47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0074b22dffe2425f90dbd82c0ed91b67)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:52.147608 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466f156f-a1ee-4b01-812a-0a7e37447a47 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:52.290762 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-466f156f-a1ee-4b01-812a-0a7e37447a47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0074b22dffe2425f90dbd82c0ed91b67', 'request_id': 'req-466f156f-a1ee-4b01-812a-0a7e37447a47'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:52.303703 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-466f156f-a1ee-4b01-812a-0a7e37447a47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0074b22dffe2425f90dbd82c0ed91b67', 'request_id': 'req-466f156f-a1ee-4b01-812a-0a7e37447a47'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:52.376936 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-466f156f-a1ee-4b01-812a-0a7e37447a47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0074b22dffe2425f90dbd82c0ed91b67', 'request_id': 'req-466f156f-a1ee-4b01-812a-0a7e37447a47'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:52.388792 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-466f156f-a1ee-4b01-812a-0a7e37447a47 admin admin] The token cache is being invalidate because project 0074b22dffe2425f90dbd82c0ed91b67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:45:52.389095 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-466f156f-a1ee-4b01-812a-0a7e37447a47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:52.389758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-466f156f-a1ee-4b01-812a-0a7e37447a47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:52.391291 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1009/2030] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:45:52 2022] DELETE /identity/v3/projects/0074b22dffe2425f90dbd82c0ed91b67 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:52.396430 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca4ae1-5fa4-4def-9933-0d48fc2e6465 service cinder] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:52.409787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca4ae1-5fa4-4def-9933-0d48fc2e6465 service cinder] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:52.487269 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e7da20-a82c-463d-84ae-088d76085f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:52.488161 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e7da20-a82c-463d-84ae-088d76085f03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:52.488481 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e7da20-a82c-463d-84ae-088d76085f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:52.488767 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e7da20-a82c-463d-84ae-088d76085f03 admin admin] PATH_INFO: `/v3/users/cf107eb0d6644b8da5e8877fb8bf9b9c` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:52.493702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e7da20-a82c-463d-84ae-088d76085f03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf107eb0d6644b8da5e8877fb8bf9b9c)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:52.493702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e7da20-a82c-463d-84ae-088d76085f03 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:52.621834 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:52.715974 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-36e7da20-a82c-463d-84ae-088d76085f03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf107eb0d6644b8da5e8877fb8bf9b9c'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:52.801564 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-36e7da20-a82c-463d-84ae-088d76085f03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf107eb0d6644b8da5e8877fb8bf9b9c'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:52.819160 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-36e7da20-a82c-463d-84ae-088d76085f03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf107eb0d6644b8da5e8877fb8bf9b9c'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:52.846922 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-36e7da20-a82c-463d-84ae-088d76085f03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf107eb0d6644b8da5e8877fb8bf9b9c'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:52.846922 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-36e7da20-a82c-463d-84ae-088d76085f03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf107eb0d6644b8da5e8877fb8bf9b9c'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:52.852073 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1022/2031] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:45:52 2022] DELETE /identity/v3/users/cf107eb0d6644b8da5e8877fb8bf9b9c => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:52.953128 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:53.448620 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466f156f-a1ee-4b01-812a-0a7e37447a47 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:53.451713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-466f156f-a1ee-4b01-812a-0a7e37447a47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:53.474245 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466f156f-a1ee-4b01-812a-0a7e37447a47 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:53.502519 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:53.539601 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5604c17-01d4-46e5-a8fb-7ad808134848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:53.543587 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5604c17-01d4-46e5-a8fb-7ad808134848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:53.543587 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5604c17-01d4-46e5-a8fb-7ad808134848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:53.543587 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5604c17-01d4-46e5-a8fb-7ad808134848 admin admin] PATH_INFO: `/v3/projects/8914bed0cb6e4431934ab70982b001c4` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:53.543587 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5604c17-01d4-46e5-a8fb-7ad808134848 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8914bed0cb6e4431934ab70982b001c4)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:53.544985 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5604c17-01d4-46e5-a8fb-7ad808134848 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:53.675856 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-d5604c17-01d4-46e5-a8fb-7ad808134848 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8914bed0cb6e4431934ab70982b001c4', 'request_id': 'req-d5604c17-01d4-46e5-a8fb-7ad808134848'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:53.682032 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-d5604c17-01d4-46e5-a8fb-7ad808134848 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8914bed0cb6e4431934ab70982b001c4', 'request_id': 'req-d5604c17-01d4-46e5-a8fb-7ad808134848'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:53.760129 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-d5604c17-01d4-46e5-a8fb-7ad808134848 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8914bed0cb6e4431934ab70982b001c4', 'request_id': 'req-d5604c17-01d4-46e5-a8fb-7ad808134848'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:53.773842 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-d5604c17-01d4-46e5-a8fb-7ad808134848 admin admin] The token cache is being invalidate because project 8914bed0cb6e4431934ab70982b001c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:45:53.774136 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-d5604c17-01d4-46e5-a8fb-7ad808134848 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:53.774817 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-d5604c17-01d4-46e5-a8fb-7ad808134848 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:53.776363 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1010/2032] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:45:53 2022] DELETE /identity/v3/projects/8914bed0cb6e4431934ab70982b001c4 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:53.781690 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db74a3-52dd-44e5-b99a-8936ab7a3ca1 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:53.782269 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db74a3-52dd-44e5-b99a-8936ab7a3ca1 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:53.782269 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db74a3-52dd-44e5-b99a-8936ab7a3ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:53.804474 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-d8db74a3-52dd-44e5-b99a-8936ab7a3ca1 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:53.805705 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-d8db74a3-52dd-44e5-b99a-8936ab7a3ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:53.840948 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:53.874487 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1023/2033] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:53.883366 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48523ff4-043c-4dce-8c4f-3d4eac65f085 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:53.883741 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48523ff4-043c-4dce-8c4f-3d4eac65f085 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:53.884025 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48523ff4-043c-4dce-8c4f-3d4eac65f085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:53.910041 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-48523ff4-043c-4dce-8c4f-3d4eac65f085 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:53.911458 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-48523ff4-043c-4dce-8c4f-3d4eac65f085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:53.920451 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1011/2034] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:53.925484 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8db74a3-52dd-44e5-b99a-8936ab7a3ca1 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:53.935730 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8db74a3-52dd-44e5-b99a-8936ab7a3ca1 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:53.947075 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006b4082-c650-4ec7-922d-ce6e11c72d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:53.948106 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b4082-c650-4ec7-922d-ce6e11c72d2f admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:53.948733 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b4082-c650-4ec7-922d-ce6e11c72d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:53.949217 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b4082-c650-4ec7-922d-ce6e11c72d2f admin admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:53.950397 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006b4082-c650-4ec7-922d-ce6e11c72d2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:45:53.950927 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006b4082-c650-4ec7-922d-ce6e11c72d2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:53.953181 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006b4082-c650-4ec7-922d-ce6e11c72d2f admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:53.980138 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1024/2035] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:45:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:45:53.981593 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48523ff4-043c-4dce-8c4f-3d4eac65f085 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:53.992058 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48523ff4-043c-4dce-8c4f-3d4eac65f085 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:53.996862 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95714a1b-4791-4ac6-9e58-0648750162fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:53.997470 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95714a1b-4791-4ac6-9e58-0648750162fb admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:53.997756 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95714a1b-4791-4ac6-9e58-0648750162fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:53.998029 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95714a1b-4791-4ac6-9e58-0648750162fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:53.998663 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95714a1b-4791-4ac6-9e58-0648750162fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:54.000713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95714a1b-4791-4ac6-9e58-0648750162fb admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:54.144897 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1012/2036] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:45:53 2022] POST /identity/v3/projects => generated 371 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:54.150372 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006b4082-c650-4ec7-922d-ce6e11c72d2f admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:54.161712 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006b4082-c650-4ec7-922d-ce6e11c72d2f admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:54.233456 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f962a-b94b-4cfe-8456-f203f5232318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:54.234083 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f962a-b94b-4cfe-8456-f203f5232318 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:54.234373 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f962a-b94b-4cfe-8456-f203f5232318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:54.234650 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f962a-b94b-4cfe-8456-f203f5232318 admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:54.235312 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821f962a-b94b-4cfe-8456-f203f5232318 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:54.238685 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821f962a-b94b-4cfe-8456-f203f5232318 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:54.356931 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1025/2037] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:45:54 2022] POST /identity/v3/users => generated 286 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:45:54.364618 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95714a1b-4791-4ac6-9e58-0648750162fb admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:54.385039 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95714a1b-4791-4ac6-9e58-0648750162fb admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:54.391019 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648d67e4-4091-4f04-94d1-f8342248cc14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:54.391720 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d67e4-4091-4f04-94d1-f8342248cc14 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:54.392057 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d67e4-4091-4f04-94d1-f8342248cc14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:54.392517 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d67e4-4091-4f04-94d1-f8342248cc14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:54.393436 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648d67e4-4091-4f04-94d1-f8342248cc14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:54.395369 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648d67e4-4091-4f04-94d1-f8342248cc14 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:54.407011 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1013/2038] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:54 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:54.411947 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f962a-b94b-4cfe-8456-f203f5232318 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:54.429905 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f962a-b94b-4cfe-8456-f203f5232318 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:54.440621 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc92d93-a2f5-4304-b4f3-0449c0f40050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:54.441717 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc92d93-a2f5-4304-b4f3-0449c0f40050 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:54.442008 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc92d93-a2f5-4304-b4f3-0449c0f40050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:54.442289 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc92d93-a2f5-4304-b4f3-0449c0f40050 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:54.442980 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc92d93-a2f5-4304-b4f3-0449c0f40050 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:54.444812 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc92d93-a2f5-4304-b4f3-0449c0f40050 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:54.458109 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1026/2039] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:54 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:54.463191 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648d67e4-4091-4f04-94d1-f8342248cc14 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:54.475554 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648d67e4-4091-4f04-94d1-f8342248cc14 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:54.480450 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee41525-c3b5-47c6-bbf1-15e75a20219e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:54.481150 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee41525-c3b5-47c6-bbf1-15e75a20219e admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:54.481427 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee41525-c3b5-47c6-bbf1-15e75a20219e admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:54.481705 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee41525-c3b5-47c6-bbf1-15e75a20219e admin admin] PATH_INFO: `/v3/projects/43149d6590a6468b83faa5f1cd7c6b59/users/1c85840c4d104b04b6ed577cce2ca2ff/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:54.499748 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee41525-c3b5-47c6-bbf1-15e75a20219e admin admin] RBAC: Authorizing `identity:create_grant(project_id=43149d6590a6468b83faa5f1cd7c6b59, user_id=1c85840c4d104b04b6ed577cce2ca2ff, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:54.502684 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee41525-c3b5-47c6-bbf1-15e75a20219e admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:54.588481 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1014/2040] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:54 2022] PUT /identity/v3/projects/43149d6590a6468b83faa5f1cd7c6b59/users/1c85840c4d104b04b6ed577cce2ca2ff/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:45:54.593913 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d06a2e-2f75-4ffa-bfba-044b1a4eaf02 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:54.594191 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d06a2e-2f75-4ffa-bfba-044b1a4eaf02 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:54.594465 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d06a2e-2f75-4ffa-bfba-044b1a4eaf02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:54.614727 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-e3d06a2e-2f75-4ffa-bfba-044b1a4eaf02 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:54.615940 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-e3d06a2e-2f75-4ffa-bfba-044b1a4eaf02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:54.679654 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1027/2041] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:54.689366 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee41525-c3b5-47c6-bbf1-15e75a20219e admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:54.692089 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-eee41525-c3b5-47c6-bbf1-15e75a20219e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:54.756295 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee41525-c3b5-47c6-bbf1-15e75a20219e admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:54.760059 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1944f86-38d0-4818-9e2d-d213a73592fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:54.760704 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1944f86-38d0-4818-9e2d-d213a73592fb service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:54.760986 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1944f86-38d0-4818-9e2d-d213a73592fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:54.761262 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1944f86-38d0-4818-9e2d-d213a73592fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:54.767459 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:54.772136 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1944f86-38d0-4818-9e2d-d213a73592fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:54.773674 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1944f86-38d0-4818-9e2d-d213a73592fb service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:54.783793 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1015/2042] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:45:56.448316 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1bbfa-7109-4c5f-9713-15b3a70285e6 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:56.448316 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1bbfa-7109-4c5f-9713-15b3a70285e6 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:56.448813 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1bbfa-7109-4c5f-9713-15b3a70285e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:56.482808 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-b6f1bbfa-7109-4c5f-9713-15b3a70285e6 None None] MFA Rules not processed for user `1c85840c4d104b04b6ed577cce2ca2ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:56.484245 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-b6f1bbfa-7109-4c5f-9713-15b3a70285e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:56.525836 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:56.562027 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1028/2043] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:45:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:56.578688 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1944f86-38d0-4818-9e2d-d213a73592fb service neutron] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:56.581104 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-f1944f86-38d0-4818-9e2d-d213a73592fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:56.646960 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1944f86-38d0-4818-9e2d-d213a73592fb service neutron] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:56.651159 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eaad4e-a46e-42da-b15b-01a4fdcf7595 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:56.651779 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eaad4e-a46e-42da-b15b-01a4fdcf7595 service cinder] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:56.652065 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eaad4e-a46e-42da-b15b-01a4fdcf7595 service cinder] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:56.652342 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eaad4e-a46e-42da-b15b-01a4fdcf7595 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:56.662974 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eaad4e-a46e-42da-b15b-01a4fdcf7595 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:56.664458 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eaad4e-a46e-42da-b15b-01a4fdcf7595 service cinder] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:56.675096 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1016/2044] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:56 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:45:58.481220 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f1bbfa-7109-4c5f-9713-15b3a70285e6 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:58.485658 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-b6f1bbfa-7109-4c5f-9713-15b3a70285e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:58.495468 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f1bbfa-7109-4c5f-9713-15b3a70285e6 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:58.499219 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf5b2a-566c-4cca-92af-9db2ca54ccaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:58.499913 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf5b2a-566c-4cca-92af-9db2ca54ccaa service cinder] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:58.500199 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf5b2a-566c-4cca-92af-9db2ca54ccaa service cinder] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:58.500520 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf5b2a-566c-4cca-92af-9db2ca54ccaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:58.502901 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-f9cf5b2a-566c-4cca-92af-9db2ca54ccaa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:58.570913 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cf5b2a-566c-4cca-92af-9db2ca54ccaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:58.572383 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cf5b2a-566c-4cca-92af-9db2ca54ccaa service cinder] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:58.583459 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1029/2045] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:58 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:00.974959 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c0caa-927d-4048-9232-110981a9e32f None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:00.975486 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c0caa-927d-4048-9232-110981a9e32f None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:00.975726 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c0caa-927d-4048-9232-110981a9e32f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:01.040137 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-6d9c0caa-927d-4048-9232-110981a9e32f None None] MFA Rules not processed for user `1c85840c4d104b04b6ed577cce2ca2ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:01.044527 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-6d9c0caa-927d-4048-9232-110981a9e32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:01.066303 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1017/2046] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:46:01.261408 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf5b2a-566c-4cca-92af-9db2ca54ccaa service cinder] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:01.263444 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-f9cf5b2a-566c-4cca-92af-9db2ca54ccaa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:01.280056 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf5b2a-566c-4cca-92af-9db2ca54ccaa service cinder] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:01.297757 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0291de5c-cba7-4f5c-9d16-04e857e1e0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:01.299584 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0291de5c-cba7-4f5c-9d16-04e857e1e0db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:01.300490 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0291de5c-cba7-4f5c-9d16-04e857e1e0db admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:01.301209 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0291de5c-cba7-4f5c-9d16-04e857e1e0db admin admin] PATH_INFO: `/v3/users/d42ed50bb9fe4701b85a413b9e23d888` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:01.303121 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0291de5c-cba7-4f5c-9d16-04e857e1e0db admin admin] RBAC: Authorizing `identity:delete_user(user_id=d42ed50bb9fe4701b85a413b9e23d888)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:01.305313 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0291de5c-cba7-4f5c-9d16-04e857e1e0db admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:02.154086 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-0291de5c-cba7-4f5c-9d16-04e857e1e0db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd42ed50bb9fe4701b85a413b9e23d888'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:02.354464 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-0291de5c-cba7-4f5c-9d16-04e857e1e0db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd42ed50bb9fe4701b85a413b9e23d888'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:02.364237 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-0291de5c-cba7-4f5c-9d16-04e857e1e0db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd42ed50bb9fe4701b85a413b9e23d888'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:02.381138 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-0291de5c-cba7-4f5c-9d16-04e857e1e0db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd42ed50bb9fe4701b85a413b9e23d888'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:02.382337 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-0291de5c-cba7-4f5c-9d16-04e857e1e0db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd42ed50bb9fe4701b85a413b9e23d888'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:02.389056 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1030/2047] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:01 2022] DELETE /identity/v3/users/d42ed50bb9fe4701b85a413b9e23d888 => generated 0 bytes in 1128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:02.395355 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ec398-3a3a-47b8-97c1-97faf52ec10a None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:02.395924 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ec398-3a3a-47b8-97c1-97faf52ec10a None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:02.396044 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ec398-3a3a-47b8-97c1-97faf52ec10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:02.418959 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-045ec398-3a3a-47b8-97c1-97faf52ec10a None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:02.420380 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-045ec398-3a3a-47b8-97c1-97faf52ec10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:02.550501 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1018/2048] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:02.560267 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0291de5c-cba7-4f5c-9d16-04e857e1e0db admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:02.563683 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-0291de5c-cba7-4f5c-9d16-04e857e1e0db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:02.626333 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:02.639774 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0291de5c-cba7-4f5c-9d16-04e857e1e0db admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:02.643559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f9ce7-8259-4f44-9e08-2ad5212ed7a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:02.644195 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f9ce7-8259-4f44-9e08-2ad5212ed7a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:02.644509 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f9ce7-8259-4f44-9e08-2ad5212ed7a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:02.644789 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f9ce7-8259-4f44-9e08-2ad5212ed7a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:02.655416 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0f9ce7-8259-4f44-9e08-2ad5212ed7a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:02.657241 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0f9ce7-8259-4f44-9e08-2ad5212ed7a3 service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:02.668748 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1031/2049] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:03.249448 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045ec398-3a3a-47b8-97c1-97faf52ec10a None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:03.251618 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-045ec398-3a3a-47b8-97c1-97faf52ec10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:03.281308 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045ec398-3a3a-47b8-97c1-97faf52ec10a None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:03.291775 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db538c90-2bbb-4b40-89ca-f5f99a2784a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:03.294850 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db538c90-2bbb-4b40-89ca-f5f99a2784a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:03.294850 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db538c90-2bbb-4b40-89ca-f5f99a2784a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:03.294850 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db538c90-2bbb-4b40-89ca-f5f99a2784a2 admin admin] PATH_INFO: `/v3/projects/233b2e73d7824d09b0539f80753731c0` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:03.297146 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db538c90-2bbb-4b40-89ca-f5f99a2784a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=233b2e73d7824d09b0539f80753731c0)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:03.301538 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db538c90-2bbb-4b40-89ca-f5f99a2784a2 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:03.475713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-db538c90-2bbb-4b40-89ca-f5f99a2784a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '233b2e73d7824d09b0539f80753731c0', 'request_id': 'req-db538c90-2bbb-4b40-89ca-f5f99a2784a2'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:03.485356 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-db538c90-2bbb-4b40-89ca-f5f99a2784a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '233b2e73d7824d09b0539f80753731c0', 'request_id': 'req-db538c90-2bbb-4b40-89ca-f5f99a2784a2'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:03.586826 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-db538c90-2bbb-4b40-89ca-f5f99a2784a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '233b2e73d7824d09b0539f80753731c0', 'request_id': 'req-db538c90-2bbb-4b40-89ca-f5f99a2784a2'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:03.598966 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-db538c90-2bbb-4b40-89ca-f5f99a2784a2 admin admin] The token cache is being invalidate because project 233b2e73d7824d09b0539f80753731c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:03.599262 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-db538c90-2bbb-4b40-89ca-f5f99a2784a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:03.599964 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-db538c90-2bbb-4b40-89ca-f5f99a2784a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:03.601530 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1019/2050] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:03 2022] DELETE /identity/v3/projects/233b2e73d7824d09b0539f80753731c0 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:03.606066 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f9ce7-8259-4f44-9e08-2ad5212ed7a3 service neutron] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:03.617365 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f9ce7-8259-4f44-9e08-2ad5212ed7a3 service neutron] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:03.697743 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972088ba-5adb-41b7-b5f8-87f7c2960972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:03.698465 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972088ba-5adb-41b7-b5f8-87f7c2960972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:03.698757 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972088ba-5adb-41b7-b5f8-87f7c2960972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:03.699605 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972088ba-5adb-41b7-b5f8-87f7c2960972 admin admin] PATH_INFO: `/v3/users/f99751fff8024584998dbc0325d27959` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:03.700600 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972088ba-5adb-41b7-b5f8-87f7c2960972 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f99751fff8024584998dbc0325d27959)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:03.702599 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972088ba-5adb-41b7-b5f8-87f7c2960972 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:04.002642 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-972088ba-5adb-41b7-b5f8-87f7c2960972 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f99751fff8024584998dbc0325d27959'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:04.130093 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-972088ba-5adb-41b7-b5f8-87f7c2960972 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f99751fff8024584998dbc0325d27959'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:04.144148 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-972088ba-5adb-41b7-b5f8-87f7c2960972 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f99751fff8024584998dbc0325d27959'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:04.155783 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db538c90-2bbb-4b40-89ca-f5f99a2784a2 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:04.159255 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-db538c90-2bbb-4b40-89ca-f5f99a2784a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:04.167548 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-972088ba-5adb-41b7-b5f8-87f7c2960972 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f99751fff8024584998dbc0325d27959'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:04.168301 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-972088ba-5adb-41b7-b5f8-87f7c2960972 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f99751fff8024584998dbc0325d27959'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:04.178918 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1032/2051] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:03 2022] DELETE /identity/v3/users/f99751fff8024584998dbc0325d27959 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:04.190826 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db538c90-2bbb-4b40-89ca-f5f99a2784a2 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:04.201588 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:04.266034 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba16527b-f030-48a9-b0cf-de58f5025326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:04.266697 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16527b-f030-48a9-b0cf-de58f5025326 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:04.266982 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16527b-f030-48a9-b0cf-de58f5025326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:04.267260 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16527b-f030-48a9-b0cf-de58f5025326 admin admin] PATH_INFO: `/v3/users/ad07185f9b8f408891ad37486724d4de` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:04.269700 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba16527b-f030-48a9-b0cf-de58f5025326 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad07185f9b8f408891ad37486724d4de)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:04.271811 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba16527b-f030-48a9-b0cf-de58f5025326 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:04.572317 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-ba16527b-f030-48a9-b0cf-de58f5025326 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad07185f9b8f408891ad37486724d4de'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:04.584296 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-ba16527b-f030-48a9-b0cf-de58f5025326 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad07185f9b8f408891ad37486724d4de'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:04.584870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-ba16527b-f030-48a9-b0cf-de58f5025326 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad07185f9b8f408891ad37486724d4de'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:04.772560 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-ba16527b-f030-48a9-b0cf-de58f5025326 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad07185f9b8f408891ad37486724d4de'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:04.787739 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-ba16527b-f030-48a9-b0cf-de58f5025326 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad07185f9b8f408891ad37486724d4de'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:04.794917 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1020/2052] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:04 2022] DELETE /identity/v3/users/ad07185f9b8f408891ad37486724d4de => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:04.800682 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22be910-f5aa-4138-88b0-72d067d41a3a None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:04.801179 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22be910-f5aa-4138-88b0-72d067d41a3a None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:04.801689 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22be910-f5aa-4138-88b0-72d067d41a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:04.817668 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba16527b-f030-48a9-b0cf-de58f5025326 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:04.821929 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-ba16527b-f030-48a9-b0cf-de58f5025326 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:04.825293 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-f22be910-f5aa-4138-88b0-72d067d41a3a None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:04.826559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-f22be910-f5aa-4138-88b0-72d067d41a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:04.899436 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1033/2053] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:04.909699 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22be910-f5aa-4138-88b0-72d067d41a3a None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:04.912443 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-f22be910-f5aa-4138-88b0-72d067d41a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:04.920392 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:04.972276 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba16527b-f030-48a9-b0cf-de58f5025326 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:04.979330 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14066a7-ec08-4921-a1c0-bcec24d9cb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:04.980189 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14066a7-ec08-4921-a1c0-bcec24d9cb03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:04.980647 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14066a7-ec08-4921-a1c0-bcec24d9cb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:04.981123 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14066a7-ec08-4921-a1c0-bcec24d9cb03 admin admin] PATH_INFO: `/v3/projects/0e9c17042b04463dbf6aae91773e209b` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:04.983298 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14066a7-ec08-4921-a1c0-bcec24d9cb03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e9c17042b04463dbf6aae91773e209b)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:04.984953 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14066a7-ec08-4921-a1c0-bcec24d9cb03 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:05.017842 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22be910-f5aa-4138-88b0-72d067d41a3a None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:05.023088 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56fa136-b432-4fd2-be17-4590bbdb7ab6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:05.023723 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56fa136-b432-4fd2-be17-4590bbdb7ab6 service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:05.024057 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56fa136-b432-4fd2-be17-4590bbdb7ab6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:05.024947 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56fa136-b432-4fd2-be17-4590bbdb7ab6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:05.036247 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56fa136-b432-4fd2-be17-4590bbdb7ab6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:05.038061 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56fa136-b432-4fd2-be17-4590bbdb7ab6 service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:05.050872 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1034/2054] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:46:05.051757 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56fa136-b432-4fd2-be17-4590bbdb7ab6 service neutron] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:05.053493 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-b56fa136-b432-4fd2-be17-4590bbdb7ab6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:05.069652 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56fa136-b432-4fd2-be17-4590bbdb7ab6 service neutron] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:05.075960 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2f76f2-9ace-4960-a0cd-074b4b220188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:05.076660 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f76f2-9ace-4960-a0cd-074b4b220188 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:05.077094 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f76f2-9ace-4960-a0cd-074b4b220188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:05.077370 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f76f2-9ace-4960-a0cd-074b4b220188 admin admin] PATH_INFO: `/v3/users/20757e37849240ef8f3209949a630117` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:05.081186 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2f76f2-9ace-4960-a0cd-074b4b220188 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20757e37849240ef8f3209949a630117)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:05.082966 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2f76f2-9ace-4960-a0cd-074b4b220188 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:05.142062 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-d14066a7-ec08-4921-a1c0-bcec24d9cb03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e9c17042b04463dbf6aae91773e209b', 'request_id': 'req-d14066a7-ec08-4921-a1c0-bcec24d9cb03'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:05.156212 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-d14066a7-ec08-4921-a1c0-bcec24d9cb03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e9c17042b04463dbf6aae91773e209b', 'request_id': 'req-d14066a7-ec08-4921-a1c0-bcec24d9cb03'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:05.337626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-d14066a7-ec08-4921-a1c0-bcec24d9cb03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e9c17042b04463dbf6aae91773e209b', 'request_id': 'req-d14066a7-ec08-4921-a1c0-bcec24d9cb03'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:05.354354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-d14066a7-ec08-4921-a1c0-bcec24d9cb03 admin admin] The token cache is being invalidate because project 0e9c17042b04463dbf6aae91773e209b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:05.354704 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-d14066a7-ec08-4921-a1c0-bcec24d9cb03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:05.356034 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-d14066a7-ec08-4921-a1c0-bcec24d9cb03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:05.362071 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1021/2055] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:04 2022] DELETE /identity/v3/projects/0e9c17042b04463dbf6aae91773e209b => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:05.365389 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8385aea-aabb-4695-afb3-4bf1939ff14b None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:05.365790 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8385aea-aabb-4695-afb3-4bf1939ff14b None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:05.366076 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8385aea-aabb-4695-afb3-4bf1939ff14b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:05.387764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-bc2f76f2-9ace-4960-a0cd-074b4b220188 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20757e37849240ef8f3209949a630117'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:05.407313 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-d8385aea-aabb-4695-afb3-4bf1939ff14b None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:05.410825 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-d8385aea-aabb-4695-afb3-4bf1939ff14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:05.488235 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-bc2f76f2-9ace-4960-a0cd-074b4b220188 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20757e37849240ef8f3209949a630117'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:05.508136 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-bc2f76f2-9ace-4960-a0cd-074b4b220188 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20757e37849240ef8f3209949a630117'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:05.514994 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1022/2056] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:05.527922 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e770b-188a-40e3-b820-fd10014be2bf None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:05.528266 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e770b-188a-40e3-b820-fd10014be2bf None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:05.528587 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e770b-188a-40e3-b820-fd10014be2bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:05.553717 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-bc2f76f2-9ace-4960-a0cd-074b4b220188 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20757e37849240ef8f3209949a630117'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:05.556843 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-bc2f76f2-9ace-4960-a0cd-074b4b220188 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20757e37849240ef8f3209949a630117'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:05.559198 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-893e770b-188a-40e3-b820-fd10014be2bf None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:05.560981 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-893e770b-188a-40e3-b820-fd10014be2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:05.575173 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1023/2057] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:05.577766 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1035/2058] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:05 2022] DELETE /identity/v3/users/20757e37849240ef8f3209949a630117 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:05.578761 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893e770b-188a-40e3-b820-fd10014be2bf None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:05.584206 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0fe7ad-bce7-428e-a936-af12ae503154 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:05.584665 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0fe7ad-bce7-428e-a936-af12ae503154 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:05.585159 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0fe7ad-bce7-428e-a936-af12ae503154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:05.595326 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893e770b-188a-40e3-b820-fd10014be2bf None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:05.636344 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-3a0fe7ad-bce7-428e-a936-af12ae503154 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:05.637645 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-3a0fe7ad-bce7-428e-a936-af12ae503154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:05.705562 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5298e38-93fe-454a-bce1-e48d9331f2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:05.706066 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5298e38-93fe-454a-bce1-e48d9331f2bb admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:05.706461 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5298e38-93fe-454a-bce1-e48d9331f2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:05.706737 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5298e38-93fe-454a-bce1-e48d9331f2bb admin admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:05.707519 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5298e38-93fe-454a-bce1-e48d9331f2bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:46:05.707726 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5298e38-93fe-454a-bce1-e48d9331f2bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:05.711762 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5298e38-93fe-454a-bce1-e48d9331f2bb admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:05.737603 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1024/2059] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:46:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:46:05.738609 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5298e38-93fe-454a-bce1-e48d9331f2bb admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:05.761969 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1036/2060] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:05.769923 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0fe7ad-bce7-428e-a936-af12ae503154 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:05.776195 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-3a0fe7ad-bce7-428e-a936-af12ae503154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:05.820033 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5298e38-93fe-454a-bce1-e48d9331f2bb admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:05.856036 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0655548a-4128-46b9-8fb4-1f053368a3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:05.856681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655548a-4128-46b9-8fb4-1f053368a3de admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:05.856959 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655548a-4128-46b9-8fb4-1f053368a3de admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:05.857231 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655548a-4128-46b9-8fb4-1f053368a3de admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:05.857903 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0655548a-4128-46b9-8fb4-1f053368a3de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:05.866502 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0655548a-4128-46b9-8fb4-1f053368a3de admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:05.982540 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0fe7ad-bce7-428e-a936-af12ae503154 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:05.982540 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20631da4-d39f-4f49-8041-3608e3efccf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:05.982540 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20631da4-d39f-4f49-8041-3608e3efccf1 service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:05.982540 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20631da4-d39f-4f49-8041-3608e3efccf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:05.984139 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20631da4-d39f-4f49-8041-3608e3efccf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:06.008618 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20631da4-d39f-4f49-8041-3608e3efccf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:06.010546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20631da4-d39f-4f49-8041-3608e3efccf1 service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:06.024161 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1037/2061] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:46:06.025096 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20631da4-d39f-4f49-8041-3608e3efccf1 service neutron] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:06.027194 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-20631da4-d39f-4f49-8041-3608e3efccf1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:06.046667 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20631da4-d39f-4f49-8041-3608e3efccf1 service neutron] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:06.055673 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26739896-b3d9-4d81-855e-7fee011e40ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:06.056620 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26739896-b3d9-4d81-855e-7fee011e40ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:06.057140 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26739896-b3d9-4d81-855e-7fee011e40ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:06.057599 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26739896-b3d9-4d81-855e-7fee011e40ee admin admin] PATH_INFO: `/v3/projects/e9111027e9024cdc8418e2a2cc39e6d8` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:06.059228 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26739896-b3d9-4d81-855e-7fee011e40ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9111027e9024cdc8418e2a2cc39e6d8)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:06.061762 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26739896-b3d9-4d81-855e-7fee011e40ee admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:06.103951 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1025/2062] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:05 2022] POST /identity/v3/projects => generated 383 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:06.107940 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0655548a-4128-46b9-8fb4-1f053368a3de admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:06.223240 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-26739896-b3d9-4d81-855e-7fee011e40ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9111027e9024cdc8418e2a2cc39e6d8', 'request_id': 'req-26739896-b3d9-4d81-855e-7fee011e40ee'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:06.231097 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-26739896-b3d9-4d81-855e-7fee011e40ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9111027e9024cdc8418e2a2cc39e6d8', 'request_id': 'req-26739896-b3d9-4d81-855e-7fee011e40ee'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:06.238400 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0655548a-4128-46b9-8fb4-1f053368a3de admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:06.336993 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b8ebc-c1c7-4bcd-a8d4-7f8d1ece1efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:06.337699 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b8ebc-c1c7-4bcd-a8d4-7f8d1ece1efb admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:06.337983 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b8ebc-c1c7-4bcd-a8d4-7f8d1ece1efb admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:06.338257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b8ebc-c1c7-4bcd-a8d4-7f8d1ece1efb admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:06.338913 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79b8ebc-c1c7-4bcd-a8d4-7f8d1ece1efb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:06.341046 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79b8ebc-c1c7-4bcd-a8d4-7f8d1ece1efb admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:06.373638 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-26739896-b3d9-4d81-855e-7fee011e40ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9111027e9024cdc8418e2a2cc39e6d8', 'request_id': 'req-26739896-b3d9-4d81-855e-7fee011e40ee'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:06.388819 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-26739896-b3d9-4d81-855e-7fee011e40ee admin admin] The token cache is being invalidate because project e9111027e9024cdc8418e2a2cc39e6d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:06.389222 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-26739896-b3d9-4d81-855e-7fee011e40ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:06.390130 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-26739896-b3d9-4d81-855e-7fee011e40ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:06.393124 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1038/2063] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:06 2022] DELETE /identity/v3/projects/e9111027e9024cdc8418e2a2cc39e6d8 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:06.397802 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26739896-b3d9-4d81-855e-7fee011e40ee admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:06.399784 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-26739896-b3d9-4d81-855e-7fee011e40ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:06.428256 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26739896-b3d9-4d81-855e-7fee011e40ee admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:06.434218 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14d06e9-ddb2-4ea1-82b6-992508e51163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:06.435027 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d06e9-ddb2-4ea1-82b6-992508e51163 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:06.435315 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d06e9-ddb2-4ea1-82b6-992508e51163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:06.436098 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d06e9-ddb2-4ea1-82b6-992508e51163 admin admin] PATH_INFO: `/v3/users/4e63bd4c90404294a81f04796820f706` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:06.437619 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14d06e9-ddb2-4ea1-82b6-992508e51163 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e63bd4c90404294a81f04796820f706)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:06.440343 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14d06e9-ddb2-4ea1-82b6-992508e51163 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:06.467353 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1026/2064] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:06 2022] POST /identity/v3/users => generated 292 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:46:06.472664 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b8ebc-c1c7-4bcd-a8d4-7f8d1ece1efb admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:06.475841 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-b79b8ebc-c1c7-4bcd-a8d4-7f8d1ece1efb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:06.513041 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b8ebc-c1c7-4bcd-a8d4-7f8d1ece1efb admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:06.522623 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d48f8-fb68-4eb0-9c97-f0e2c598daaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:06.523363 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d48f8-fb68-4eb0-9c97-f0e2c598daaf admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:06.525140 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d48f8-fb68-4eb0-9c97-f0e2c598daaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:06.525140 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d48f8-fb68-4eb0-9c97-f0e2c598daaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:06.526519 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737d48f8-fb68-4eb0-9c97-f0e2c598daaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:06.529274 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737d48f8-fb68-4eb0-9c97-f0e2c598daaf admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:06.542138 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1027/2065] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:06 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:46:06.547788 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d48f8-fb68-4eb0-9c97-f0e2c598daaf admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:06.560616 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d48f8-fb68-4eb0-9c97-f0e2c598daaf admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:06.566544 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd280c6-ee89-42b1-aaaf-8354d8d16f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:06.567329 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd280c6-ee89-42b1-aaaf-8354d8d16f15 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:06.567739 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd280c6-ee89-42b1-aaaf-8354d8d16f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:06.568116 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd280c6-ee89-42b1-aaaf-8354d8d16f15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:06.570189 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd280c6-ee89-42b1-aaaf-8354d8d16f15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:06.572857 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd280c6-ee89-42b1-aaaf-8354d8d16f15 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:06.592965 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1028/2066] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:06 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:46:06.594090 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd280c6-ee89-42b1-aaaf-8354d8d16f15 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:06.615889 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd280c6-ee89-42b1-aaaf-8354d8d16f15 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:06.621679 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd1b396-ed92-4c68-b449-35c686a13132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:06.622827 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1b396-ed92-4c68-b449-35c686a13132 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:06.622827 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1b396-ed92-4c68-b449-35c686a13132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:06.623117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1b396-ed92-4c68-b449-35c686a13132 admin admin] PATH_INFO: `/v3/projects/39fcd5de93c744388a2f41a0daf959cd/users/2e4ff010ef794b3c972488e781e0b19a/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:06.646600 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd1b396-ed92-4c68-b449-35c686a13132 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39fcd5de93c744388a2f41a0daf959cd, user_id=2e4ff010ef794b3c972488e781e0b19a, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:06.650435 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd1b396-ed92-4c68-b449-35c686a13132 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:06.769548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:06.777254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1029/2067] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:06 2022] PUT /identity/v3/projects/39fcd5de93c744388a2f41a0daf959cd/users/2e4ff010ef794b3c972488e781e0b19a/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:06.778953 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd1b396-ed92-4c68-b449-35c686a13132 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:06.784144 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-4fd1b396-ed92-4c68-b449-35c686a13132 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:06.808967 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-e14d06e9-ddb2-4ea1-82b6-992508e51163 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e63bd4c90404294a81f04796820f706'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:06.903138 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-e14d06e9-ddb2-4ea1-82b6-992508e51163 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e63bd4c90404294a81f04796820f706'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:06.903516 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd1b396-ed92-4c68-b449-35c686a13132 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:06.913545 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a8791-566c-4b23-b853-ede82fc1a3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:06.914203 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a8791-566c-4b23-b853-ede82fc1a3d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:06.914545 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a8791-566c-4b23-b853-ede82fc1a3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:06.914824 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a8791-566c-4b23-b853-ede82fc1a3d7 admin admin] PATH_INFO: `/v3/projects/84a5522913ee448bb8593f90cb7af0f6` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:06.915383 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-e14d06e9-ddb2-4ea1-82b6-992508e51163 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e63bd4c90404294a81f04796820f706'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:06.919036 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a8791-566c-4b23-b853-ede82fc1a3d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84a5522913ee448bb8593f90cb7af0f6)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:06.921073 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a8791-566c-4b23-b853-ede82fc1a3d7 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:06.942329 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-e14d06e9-ddb2-4ea1-82b6-992508e51163 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e63bd4c90404294a81f04796820f706'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:06.943423 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-e14d06e9-ddb2-4ea1-82b6-992508e51163 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e63bd4c90404294a81f04796820f706'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:06.969005 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1039/2068] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:06 2022] DELETE /identity/v3/users/4e63bd4c90404294a81f04796820f706 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:06.973498 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9796fe7-3d96-4157-b94e-c0f15fe89f26 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:06.974206 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9796fe7-3d96-4157-b94e-c0f15fe89f26 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:06.976375 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9796fe7-3d96-4157-b94e-c0f15fe89f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:07.069032 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-c9796fe7-3d96-4157-b94e-c0f15fe89f26 None None] MFA Rules not processed for user `2e4ff010ef794b3c972488e781e0b19a`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:07.070491 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-c9796fe7-3d96-4157-b94e-c0f15fe89f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:07.135162 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-711a8791-566c-4b23-b853-ede82fc1a3d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84a5522913ee448bb8593f90cb7af0f6', 'request_id': 'req-711a8791-566c-4b23-b853-ede82fc1a3d7'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:07.147985 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-711a8791-566c-4b23-b853-ede82fc1a3d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84a5522913ee448bb8593f90cb7af0f6', 'request_id': 'req-711a8791-566c-4b23-b853-ede82fc1a3d7'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:07.157959 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1040/2069] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:07.164909 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:07.166988 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9796fe7-3d96-4157-b94e-c0f15fe89f26 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:07.168680 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-c9796fe7-3d96-4157-b94e-c0f15fe89f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:07.222653 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-711a8791-566c-4b23-b853-ede82fc1a3d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84a5522913ee448bb8593f90cb7af0f6', 'request_id': 'req-711a8791-566c-4b23-b853-ede82fc1a3d7'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:07.250123 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-711a8791-566c-4b23-b853-ede82fc1a3d7 admin admin] The token cache is being invalidate because project 84a5522913ee448bb8593f90cb7af0f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:07.250743 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-711a8791-566c-4b23-b853-ede82fc1a3d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:07.252081 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-711a8791-566c-4b23-b853-ede82fc1a3d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:07.257014 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1030/2070] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:06 2022] DELETE /identity/v3/projects/84a5522913ee448bb8593f90cb7af0f6 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:07.277541 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a8791-566c-4b23-b853-ede82fc1a3d7 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:07.282847 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-711a8791-566c-4b23-b853-ede82fc1a3d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:07.313474 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9796fe7-3d96-4157-b94e-c0f15fe89f26 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:07.438099 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a8791-566c-4b23-b853-ede82fc1a3d7 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:07.446637 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a42aec-5aa8-4a30-aa71-e2843730ef02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:07.447356 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a42aec-5aa8-4a30-aa71-e2843730ef02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:07.447647 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a42aec-5aa8-4a30-aa71-e2843730ef02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:07.448463 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a42aec-5aa8-4a30-aa71-e2843730ef02 admin admin] PATH_INFO: `/v3/users/54afd1b4b1c64fb29c0c76b880c52ac3` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:07.449689 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a42aec-5aa8-4a30-aa71-e2843730ef02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54afd1b4b1c64fb29c0c76b880c52ac3)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:07.451563 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a42aec-5aa8-4a30-aa71-e2843730ef02 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:07.481422 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10340d4f-ee86-4c75-8fd3-0a9676e13cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:07.482136 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10340d4f-ee86-4c75-8fd3-0a9676e13cce admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:07.482477 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10340d4f-ee86-4c75-8fd3-0a9676e13cce admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:07.482769 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10340d4f-ee86-4c75-8fd3-0a9676e13cce admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:07.483603 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10340d4f-ee86-4c75-8fd3-0a9676e13cce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:07.485858 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10340d4f-ee86-4c75-8fd3-0a9676e13cce admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:07.654606 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1041/2071] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:07 2022] POST /identity/v3/projects => generated 385 bytes in 491 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:07.659235 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10340d4f-ee86-4c75-8fd3-0a9676e13cce admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:07.662043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-10340d4f-ee86-4c75-8fd3-0a9676e13cce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:07.763492 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10340d4f-ee86-4c75-8fd3-0a9676e13cce admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:07.784310 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa32dc5-611d-40ac-931c-8e3beac3b260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:07.784310 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa32dc5-611d-40ac-931c-8e3beac3b260 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:07.784310 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa32dc5-611d-40ac-931c-8e3beac3b260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:07.784310 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa32dc5-611d-40ac-931c-8e3beac3b260 admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:07.785065 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa32dc5-611d-40ac-931c-8e3beac3b260 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:07.785065 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa32dc5-611d-40ac-931c-8e3beac3b260 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:08.006232 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-b7a42aec-5aa8-4a30-aa71-e2843730ef02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54afd1b4b1c64fb29c0c76b880c52ac3'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:08.021110 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-b7a42aec-5aa8-4a30-aa71-e2843730ef02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54afd1b4b1c64fb29c0c76b880c52ac3'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:08.027870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-b7a42aec-5aa8-4a30-aa71-e2843730ef02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54afd1b4b1c64fb29c0c76b880c52ac3'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:08.035141 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1042/2072] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:07 2022] POST /identity/v3/users => generated 299 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:08.036478 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa32dc5-611d-40ac-931c-8e3beac3b260 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:08.038963 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-0fa32dc5-611d-40ac-931c-8e3beac3b260 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:08.056773 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa32dc5-611d-40ac-931c-8e3beac3b260 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:08.063491 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7799e6-f2b6-4513-87a3-70a4127c942a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:08.064764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7799e6-f2b6-4513-87a3-70a4127c942a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:08.065207 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7799e6-f2b6-4513-87a3-70a4127c942a admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:08.065714 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7799e6-f2b6-4513-87a3-70a4127c942a admin admin] PATH_INFO: `/v3/projects/b28537a07c1542a580896fcec2e52fff` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:08.067626 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7799e6-f2b6-4513-87a3-70a4127c942a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b28537a07c1542a580896fcec2e52fff)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:08.071593 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7799e6-f2b6-4513-87a3-70a4127c942a admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:08.118401 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-b7a42aec-5aa8-4a30-aa71-e2843730ef02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54afd1b4b1c64fb29c0c76b880c52ac3'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:08.139268 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-b7a42aec-5aa8-4a30-aa71-e2843730ef02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54afd1b4b1c64fb29c0c76b880c52ac3'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:08.150670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1031/2073] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:07 2022] DELETE /identity/v3/users/54afd1b4b1c64fb29c0c76b880c52ac3 => generated 0 bytes in 875 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:08.150670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a42aec-5aa8-4a30-aa71-e2843730ef02 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:08.152186 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-b7a42aec-5aa8-4a30-aa71-e2843730ef02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:08.265046 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-3d7799e6-f2b6-4513-87a3-70a4127c942a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b28537a07c1542a580896fcec2e52fff', 'request_id': 'req-3d7799e6-f2b6-4513-87a3-70a4127c942a'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:08.277005 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-3d7799e6-f2b6-4513-87a3-70a4127c942a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b28537a07c1542a580896fcec2e52fff', 'request_id': 'req-3d7799e6-f2b6-4513-87a3-70a4127c942a'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:08.277267 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a42aec-5aa8-4a30-aa71-e2843730ef02 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:08.345438 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11651f02-f7a2-4013-a8e0-fd03a5c763e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:08.346361 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11651f02-f7a2-4013-a8e0-fd03a5c763e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:08.346660 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11651f02-f7a2-4013-a8e0-fd03a5c763e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:08.346935 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11651f02-f7a2-4013-a8e0-fd03a5c763e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:08.347597 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11651f02-f7a2-4013-a8e0-fd03a5c763e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:08.349310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11651f02-f7a2-4013-a8e0-fd03a5c763e8 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:08.361821 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1032/2074] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:08 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:08.366001 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11651f02-f7a2-4013-a8e0-fd03a5c763e8 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:08.376847 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11651f02-f7a2-4013-a8e0-fd03a5c763e8 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:08.382887 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2900a335-326c-48f0-91ae-7364b525dd46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:08.383879 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2900a335-326c-48f0-91ae-7364b525dd46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:08.384145 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2900a335-326c-48f0-91ae-7364b525dd46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:08.384522 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2900a335-326c-48f0-91ae-7364b525dd46 admin admin] PATH_INFO: `/v3/projects/2be1f542a9df4f37be9bf2c065c6dda3/users/16b0404333fa45b0b4645169ab39ff66/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:08.406173 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2900a335-326c-48f0-91ae-7364b525dd46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2be1f542a9df4f37be9bf2c065c6dda3, user_id=16b0404333fa45b0b4645169ab39ff66, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:08.411200 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2900a335-326c-48f0-91ae-7364b525dd46 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:08.411598 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-3d7799e6-f2b6-4513-87a3-70a4127c942a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b28537a07c1542a580896fcec2e52fff', 'request_id': 'req-3d7799e6-f2b6-4513-87a3-70a4127c942a'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:08.429906 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-3d7799e6-f2b6-4513-87a3-70a4127c942a admin admin] The token cache is being invalidate because project b28537a07c1542a580896fcec2e52fff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:08.430305 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-3d7799e6-f2b6-4513-87a3-70a4127c942a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:08.435740 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-3d7799e6-f2b6-4513-87a3-70a4127c942a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:08.440027 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1043/2075] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:08 2022] DELETE /identity/v3/projects/b28537a07c1542a580896fcec2e52fff => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:08.446429 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f6150-1456-4015-8e83-348f3d76edc2 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:08.446786 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f6150-1456-4015-8e83-348f3d76edc2 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:08.447282 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f6150-1456-4015-8e83-348f3d76edc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:08.491707 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-fc7f6150-1456-4015-8e83-348f3d76edc2 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:08.492932 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-fc7f6150-1456-4015-8e83-348f3d76edc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:08.504938 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1044/2076] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:08.518527 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfbe3c4-11a4-4795-90b9-9c43158f27a9 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:08.538792 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfbe3c4-11a4-4795-90b9-9c43158f27a9 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:08.539822 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfbe3c4-11a4-4795-90b9-9c43158f27a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:08.550319 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1033/2077] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:08 2022] PUT /identity/v3/projects/2be1f542a9df4f37be9bf2c065c6dda3/users/16b0404333fa45b0b4645169ab39ff66/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:46:08.558378 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2900a335-326c-48f0-91ae-7364b525dd46 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:08.563080 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-2900a335-326c-48f0-91ae-7364b525dd46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:08.585909 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-cbfbe3c4-11a4-4795-90b9-9c43158f27a9 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:08.590048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-cbfbe3c4-11a4-4795-90b9-9c43158f27a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:08.711307 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1045/2078] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:08.715343 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfbe3c4-11a4-4795-90b9-9c43158f27a9 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:08.732070 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2900a335-326c-48f0-91ae-7364b525dd46 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:08.735671 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfbe3c4-11a4-4795-90b9-9c43158f27a9 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:08.738553 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230224b4-1353-43cb-bf6b-a409413a4b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:08.739510 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230224b4-1353-43cb-bf6b-a409413a4b2c admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:08.740240 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230224b4-1353-43cb-bf6b-a409413a4b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:08.740774 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230224b4-1353-43cb-bf6b-a409413a4b2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:08.741773 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230224b4-1353-43cb-bf6b-a409413a4b2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:08.743954 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230224b4-1353-43cb-bf6b-a409413a4b2c admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:08.757052 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb1409-da5d-4e8c-a7a4-e298969ff3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:08.757052 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb1409-da5d-4e8c-a7a4-e298969ff3ae admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:08.757052 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb1409-da5d-4e8c-a7a4-e298969ff3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:08.757052 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb1409-da5d-4e8c-a7a4-e298969ff3ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:08.757880 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cb1409-da5d-4e8c-a7a4-e298969ff3ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:46:08.757880 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cb1409-da5d-4e8c-a7a4-e298969ff3ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:08.757880 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cb1409-da5d-4e8c-a7a4-e298969ff3ae admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:08.760509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1034/2079] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:08 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:08.766514 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230224b4-1353-43cb-bf6b-a409413a4b2c admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:08.780736 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1046/2080] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:46:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:46:08.783532 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230224b4-1353-43cb-bf6b-a409413a4b2c admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:08.788686 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb1409-da5d-4e8c-a7a4-e298969ff3ae admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:08.793031 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e3f182-d183-4264-9e28-90c5e9d06d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:08.802802 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3f182-d183-4264-9e28-90c5e9d06d76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:08.802802 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3f182-d183-4264-9e28-90c5e9d06d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:08.802802 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3f182-d183-4264-9e28-90c5e9d06d76 admin admin] PATH_INFO: `/v3/projects/2be1f542a9df4f37be9bf2c065c6dda3/users/16b0404333fa45b0b4645169ab39ff66/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:08.802802 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e3f182-d183-4264-9e28-90c5e9d06d76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2be1f542a9df4f37be9bf2c065c6dda3, user_id=16b0404333fa45b0b4645169ab39ff66, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:08.803611 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e3f182-d183-4264-9e28-90c5e9d06d76 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:08.814168 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb1409-da5d-4e8c-a7a4-e298969ff3ae admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:08.815697 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1035/2081] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:08 2022] PUT /identity/v3/projects/2be1f542a9df4f37be9bf2c065c6dda3/users/16b0404333fa45b0b4645169ab39ff66/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:46:08.847821 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd53d0e-3516-4fc5-b465-25af6317acfb None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:08.847821 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd53d0e-3516-4fc5-b465-25af6317acfb None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:08.847821 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd53d0e-3516-4fc5-b465-25af6317acfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:08.897094 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:08.915781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-2fd53d0e-3516-4fc5-b465-25af6317acfb None None] MFA Rules not processed for user `16b0404333fa45b0b4645169ab39ff66`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:08.917569 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-2fd53d0e-3516-4fc5-b465-25af6317acfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:08.948070 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc22f6f-55bc-4cea-b861-15b4464fbd12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:08.948722 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc22f6f-55bc-4cea-b861-15b4464fbd12 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:08.949111 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc22f6f-55bc-4cea-b861-15b4464fbd12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:08.949368 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc22f6f-55bc-4cea-b861-15b4464fbd12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:08.950125 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc22f6f-55bc-4cea-b861-15b4464fbd12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:08.952567 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc22f6f-55bc-4cea-b861-15b4464fbd12 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:09.067299 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1036/2082] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:08 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:09.070665 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1047/2083] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:08 2022] POST /identity/v3/projects => generated 379 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:09.075381 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd53d0e-3516-4fc5-b465-25af6317acfb None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:09.101256 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc22f6f-55bc-4cea-b861-15b4464fbd12 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:09.106967 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd53d0e-3516-4fc5-b465-25af6317acfb None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:09.109935 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-bbc22f6f-55bc-4cea-b861-15b4464fbd12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:09.249402 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c8767-5ecb-4722-b20e-a4624612b6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:09.250012 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c8767-5ecb-4722-b20e-a4624612b6cf admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:09.250304 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c8767-5ecb-4722-b20e-a4624612b6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:09.250586 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c8767-5ecb-4722-b20e-a4624612b6cf admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:09.251425 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762c8767-5ecb-4722-b20e-a4624612b6cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:09.254524 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762c8767-5ecb-4722-b20e-a4624612b6cf admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:09.266057 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc22f6f-55bc-4cea-b861-15b4464fbd12 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:09.269720 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:09.273823 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74707e66-c111-4669-9c23-9db3e8d7ab41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:09.274898 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74707e66-c111-4669-9c23-9db3e8d7ab41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:09.275358 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74707e66-c111-4669-9c23-9db3e8d7ab41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:09.275686 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74707e66-c111-4669-9c23-9db3e8d7ab41 admin admin] PATH_INFO: `/v3/projects/417a2c54a3a048bcaa6cdc49fd08ada9` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:09.277262 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74707e66-c111-4669-9c23-9db3e8d7ab41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=417a2c54a3a048bcaa6cdc49fd08ada9)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:09.279604 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74707e66-c111-4669-9c23-9db3e8d7ab41 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:09.418032 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1037/2084] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:09 2022] POST /identity/v3/users => generated 290 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:46:09.420748 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c8767-5ecb-4722-b20e-a4624612b6cf admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:09.424419 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-762c8767-5ecb-4722-b20e-a4624612b6cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:09.459256 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-74707e66-c111-4669-9c23-9db3e8d7ab41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '417a2c54a3a048bcaa6cdc49fd08ada9', 'request_id': 'req-74707e66-c111-4669-9c23-9db3e8d7ab41'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:09.470215 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-74707e66-c111-4669-9c23-9db3e8d7ab41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '417a2c54a3a048bcaa6cdc49fd08ada9', 'request_id': 'req-74707e66-c111-4669-9c23-9db3e8d7ab41'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:09.512436 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c8767-5ecb-4722-b20e-a4624612b6cf admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:09.566082 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-74707e66-c111-4669-9c23-9db3e8d7ab41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '417a2c54a3a048bcaa6cdc49fd08ada9', 'request_id': 'req-74707e66-c111-4669-9c23-9db3e8d7ab41'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:09.573559 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a70020-35f8-455d-91b9-e2f9bd9cd27e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:09.581500 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a70020-35f8-455d-91b9-e2f9bd9cd27e service cinder] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:09.581500 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a70020-35f8-455d-91b9-e2f9bd9cd27e service cinder] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:09.581500 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a70020-35f8-455d-91b9-e2f9bd9cd27e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:09.581500 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-63a70020-35f8-455d-91b9-e2f9bd9cd27e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:09.593386 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-74707e66-c111-4669-9c23-9db3e8d7ab41 admin admin] The token cache is being invalidate because project 417a2c54a3a048bcaa6cdc49fd08ada9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:09.593789 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-74707e66-c111-4669-9c23-9db3e8d7ab41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:09.594800 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-74707e66-c111-4669-9c23-9db3e8d7ab41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:09.597159 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1048/2085] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:09 2022] DELETE /identity/v3/projects/417a2c54a3a048bcaa6cdc49fd08ada9 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:09.598541 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74707e66-c111-4669-9c23-9db3e8d7ab41 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:09.600931 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-74707e66-c111-4669-9c23-9db3e8d7ab41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:09.656694 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a70020-35f8-455d-91b9-e2f9bd9cd27e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:09.658742 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a70020-35f8-455d-91b9-e2f9bd9cd27e service cinder] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:09.681000 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1038/2086] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:09.682081 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a70020-35f8-455d-91b9-e2f9bd9cd27e service cinder] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:09.690411 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74707e66-c111-4669-9c23-9db3e8d7ab41 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:09.696899 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a70020-35f8-455d-91b9-e2f9bd9cd27e service cinder] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:09.698647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d6929-8ee7-4d59-8d9e-91cc248a4527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:09.699977 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d6929-8ee7-4d59-8d9e-91cc248a4527 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:09.700375 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d6929-8ee7-4d59-8d9e-91cc248a4527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:09.701147 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d6929-8ee7-4d59-8d9e-91cc248a4527 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:09.702186 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867d6929-8ee7-4d59-8d9e-91cc248a4527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:09.703009 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3c6c7-f7d3-4bb3-bf61-d6090195c251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:09.703700 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3c6c7-f7d3-4bb3-bf61-d6090195c251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:09.704034 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3c6c7-f7d3-4bb3-bf61-d6090195c251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:09.705777 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3c6c7-f7d3-4bb3-bf61-d6090195c251 admin admin] PATH_INFO: `/v3/users/1659b79bf7594e18b84e23f39b0917c8` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:09.706028 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867d6929-8ee7-4d59-8d9e-91cc248a4527 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:09.706658 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e3c6c7-f7d3-4bb3-bf61-d6090195c251 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1659b79bf7594e18b84e23f39b0917c8)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:09.708731 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e3c6c7-f7d3-4bb3-bf61-d6090195c251 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:09.722104 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1049/2087] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:09 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:09.728143 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d6929-8ee7-4d59-8d9e-91cc248a4527 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:09.746283 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d6929-8ee7-4d59-8d9e-91cc248a4527 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:09.755221 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15741f8e-bcd2-4494-ac9f-b28a966a7fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:09.756055 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15741f8e-bcd2-4494-ac9f-b28a966a7fd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:09.756563 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15741f8e-bcd2-4494-ac9f-b28a966a7fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:09.757025 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15741f8e-bcd2-4494-ac9f-b28a966a7fd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:09.758970 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15741f8e-bcd2-4494-ac9f-b28a966a7fd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:09.761796 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15741f8e-bcd2-4494-ac9f-b28a966a7fd8 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:09.781481 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1050/2088] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:09 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:46:09.790898 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15741f8e-bcd2-4494-ac9f-b28a966a7fd8 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:09.806992 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15741f8e-bcd2-4494-ac9f-b28a966a7fd8 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:09.815510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19709cf-43c0-4cf3-822e-96fed3122f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:09.821241 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19709cf-43c0-4cf3-822e-96fed3122f8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:09.821241 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19709cf-43c0-4cf3-822e-96fed3122f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:09.821241 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19709cf-43c0-4cf3-822e-96fed3122f8d admin admin] PATH_INFO: `/v3/projects/ec7531f7ea474d88a50104ea32362867/users/77cb75e0dea3422b9d31e740636c5641/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:09.844832 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19709cf-43c0-4cf3-822e-96fed3122f8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec7531f7ea474d88a50104ea32362867, user_id=77cb75e0dea3422b9d31e740636c5641, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:09.850075 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19709cf-43c0-4cf3-822e-96fed3122f8d admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:09.999991 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1051/2089] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:09 2022] PUT /identity/v3/projects/ec7531f7ea474d88a50104ea32362867/users/77cb75e0dea3422b9d31e740636c5641/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:10.010526 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aed22e-4a4e-4f5e-b02c-ad2acfa754e1 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:10.011021 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aed22e-4a4e-4f5e-b02c-ad2acfa754e1 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:10.011298 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aed22e-4a4e-4f5e-b02c-ad2acfa754e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:10.030337 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-d2e3c6c7-f7d3-4bb3-bf61-d6090195c251 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1659b79bf7594e18b84e23f39b0917c8'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:10.045088 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-d2e3c6c7-f7d3-4bb3-bf61-d6090195c251 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1659b79bf7594e18b84e23f39b0917c8'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:10.046582 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-d2e3c6c7-f7d3-4bb3-bf61-d6090195c251 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1659b79bf7594e18b84e23f39b0917c8'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:10.053394 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-53aed22e-4a4e-4f5e-b02c-ad2acfa754e1 None None] MFA Rules not processed for user `77cb75e0dea3422b9d31e740636c5641`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:10.054861 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-53aed22e-4a4e-4f5e-b02c-ad2acfa754e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:10.151217 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1052/2090] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:10.156077 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aed22e-4a4e-4f5e-b02c-ad2acfa754e1 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:10.157706 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-53aed22e-4a4e-4f5e-b02c-ad2acfa754e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:10.159631 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-d2e3c6c7-f7d3-4bb3-bf61-d6090195c251 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1659b79bf7594e18b84e23f39b0917c8'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:10.179014 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-d2e3c6c7-f7d3-4bb3-bf61-d6090195c251 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1659b79bf7594e18b84e23f39b0917c8'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:10.198722 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1039/2091] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:09 2022] DELETE /identity/v3/users/1659b79bf7594e18b84e23f39b0917c8 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:10.238824 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aed22e-4a4e-4f5e-b02c-ad2acfa754e1 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:10.341584 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a7f2c-e2ec-416a-a4b7-4ea0837f89d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:10.342372 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a7f2c-e2ec-416a-a4b7-4ea0837f89d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:10.342760 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a7f2c-e2ec-416a-a4b7-4ea0837f89d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:10.343130 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a7f2c-e2ec-416a-a4b7-4ea0837f89d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:10.343878 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0a7f2c-e2ec-416a-a4b7-4ea0837f89d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:10.347365 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0a7f2c-e2ec-416a-a4b7-4ea0837f89d7 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:10.479680 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1053/2092] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:10 2022] POST /identity/v3/projects => generated 379 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:46:10.484728 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3c6c7-f7d3-4bb3-bf61-d6090195c251 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:10.504831 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3c6c7-f7d3-4bb3-bf61-d6090195c251 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:10.577141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81659580-6c3b-4c32-bf7c-c316427b4263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:10.577880 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81659580-6c3b-4c32-bf7c-c316427b4263 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:10.578243 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81659580-6c3b-4c32-bf7c-c316427b4263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:10.578649 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81659580-6c3b-4c32-bf7c-c316427b4263 admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:10.579509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81659580-6c3b-4c32-bf7c-c316427b4263 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:10.583614 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81659580-6c3b-4c32-bf7c-c316427b4263 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:10.678416 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1040/2093] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:10 2022] POST /identity/v3/users => generated 296 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:10.680306 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a7f2c-e2ec-416a-a4b7-4ea0837f89d7 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:10.691662 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a7f2c-e2ec-416a-a4b7-4ea0837f89d7 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:10.696734 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77891dfa-19f2-436b-bb19-4b0b8485d3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:10.697370 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77891dfa-19f2-436b-bb19-4b0b8485d3dd admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:10.698148 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77891dfa-19f2-436b-bb19-4b0b8485d3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:10.698439 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77891dfa-19f2-436b-bb19-4b0b8485d3dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:10.699113 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77891dfa-19f2-436b-bb19-4b0b8485d3dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:10.700798 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77891dfa-19f2-436b-bb19-4b0b8485d3dd admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:10.712924 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1054/2094] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:10.717659 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81659580-6c3b-4c32-bf7c-c316427b4263 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:10.729519 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81659580-6c3b-4c32-bf7c-c316427b4263 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:10.734488 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1344ed-0ae0-4162-a976-c1fc56286b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:10.735171 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1344ed-0ae0-4162-a976-c1fc56286b08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:10.735484 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1344ed-0ae0-4162-a976-c1fc56286b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:10.735765 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1344ed-0ae0-4162-a976-c1fc56286b08 admin admin] PATH_INFO: `/v3/projects/1afe0bbede2a4abebea9cfa7eedbb5ff/users/04f89623c2cd45f6865af11229eb38fa/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:10.756254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1344ed-0ae0-4162-a976-c1fc56286b08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1afe0bbede2a4abebea9cfa7eedbb5ff, user_id=04f89623c2cd45f6865af11229eb38fa, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:10.759388 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1344ed-0ae0-4162-a976-c1fc56286b08 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:10.774936 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77891dfa-19f2-436b-bb19-4b0b8485d3dd admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:10.776931 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-77891dfa-19f2-436b-bb19-4b0b8485d3dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:10.803998 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77891dfa-19f2-436b-bb19-4b0b8485d3dd admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:10.819156 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a1060-2ab1-4e04-a5a7-180aa5b26e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:10.821299 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a1060-2ab1-4e04-a5a7-180aa5b26e18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:10.821778 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a1060-2ab1-4e04-a5a7-180aa5b26e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:10.822255 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a1060-2ab1-4e04-a5a7-180aa5b26e18 admin admin] PATH_INFO: `/v3/projects/a12e188c65f64ca2b2afab18bc9a76ed` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:10.824661 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168a1060-2ab1-4e04-a5a7-180aa5b26e18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a12e188c65f64ca2b2afab18bc9a76ed)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:10.829256 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168a1060-2ab1-4e04-a5a7-180aa5b26e18 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:10.903257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:10.909798 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1041/2095] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:10 2022] PUT /identity/v3/projects/1afe0bbede2a4abebea9cfa7eedbb5ff/users/04f89623c2cd45f6865af11229eb38fa/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:10.914565 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1344ed-0ae0-4162-a976-c1fc56286b08 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:10.980874 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-168a1060-2ab1-4e04-a5a7-180aa5b26e18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a12e188c65f64ca2b2afab18bc9a76ed', 'request_id': 'req-168a1060-2ab1-4e04-a5a7-180aa5b26e18'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:10.989178 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-168a1060-2ab1-4e04-a5a7-180aa5b26e18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a12e188c65f64ca2b2afab18bc9a76ed', 'request_id': 'req-168a1060-2ab1-4e04-a5a7-180aa5b26e18'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:11.013486 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1344ed-0ae0-4162-a976-c1fc56286b08 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:11.087428 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f10ee0-be7e-437f-acff-0883213c839e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:11.087969 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f10ee0-be7e-437f-acff-0883213c839e admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:11.088253 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f10ee0-be7e-437f-acff-0883213c839e admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:11.088573 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f10ee0-be7e-437f-acff-0883213c839e admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:11.089248 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f10ee0-be7e-437f-acff-0883213c839e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:11.090994 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f10ee0-be7e-437f-acff-0883213c839e admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:11.108961 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1042/2096] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:10 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:46:11.109260 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-168a1060-2ab1-4e04-a5a7-180aa5b26e18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a12e188c65f64ca2b2afab18bc9a76ed', 'request_id': 'req-168a1060-2ab1-4e04-a5a7-180aa5b26e18'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:11.111084 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f10ee0-be7e-437f-acff-0883213c839e admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:11.122543 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f10ee0-be7e-437f-acff-0883213c839e admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:11.134365 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5cb2d5-f4b5-46bf-bb2b-43176c74ad8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:11.141893 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5cb2d5-f4b5-46bf-bb2b-43176c74ad8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:11.141893 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5cb2d5-f4b5-46bf-bb2b-43176c74ad8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:11.141893 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5cb2d5-f4b5-46bf-bb2b-43176c74ad8f admin admin] PATH_INFO: `/v3/projects/1afe0bbede2a4abebea9cfa7eedbb5ff/users/04f89623c2cd45f6865af11229eb38fa/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:11.141893 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5cb2d5-f4b5-46bf-bb2b-43176c74ad8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1afe0bbede2a4abebea9cfa7eedbb5ff, user_id=04f89623c2cd45f6865af11229eb38fa, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:11.141893 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5cb2d5-f4b5-46bf-bb2b-43176c74ad8f admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:11.147694 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-168a1060-2ab1-4e04-a5a7-180aa5b26e18 admin admin] The token cache is being invalidate because project a12e188c65f64ca2b2afab18bc9a76ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:11.147994 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-168a1060-2ab1-4e04-a5a7-180aa5b26e18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:11.153045 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-168a1060-2ab1-4e04-a5a7-180aa5b26e18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:11.170354 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1055/2097] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:10 2022] DELETE /identity/v3/projects/a12e188c65f64ca2b2afab18bc9a76ed => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:46:11.171921 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e7c1b-84c9-4ce8-bad0-c78b45fed960 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:11.172214 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e7c1b-84c9-4ce8-bad0-c78b45fed960 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:11.172518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e7c1b-84c9-4ce8-bad0-c78b45fed960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:11.173848 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1043/2098] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:11 2022] PUT /identity/v3/projects/1afe0bbede2a4abebea9cfa7eedbb5ff/users/04f89623c2cd45f6865af11229eb38fa/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:11.192619 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311f929-c911-40b5-ac09-8f13e60b2139 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:11.192935 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311f929-c911-40b5-ac09-8f13e60b2139 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:11.193178 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311f929-c911-40b5-ac09-8f13e60b2139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:11.230571 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-724e7c1b-84c9-4ce8-bad0-c78b45fed960 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:11.232087 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-724e7c1b-84c9-4ce8-bad0-c78b45fed960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:11.246686 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-e311f929-c911-40b5-ac09-8f13e60b2139 None None] MFA Rules not processed for user `04f89623c2cd45f6865af11229eb38fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:11.248067 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-e311f929-c911-40b5-ac09-8f13e60b2139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:11.301137 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:11.335012 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1056/2099] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:11.349542 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fd554-5abd-44c8-a65d-5b2ce5ea5dc0 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:11.349923 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fd554-5abd-44c8-a65d-5b2ce5ea5dc0 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:11.350273 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fd554-5abd-44c8-a65d-5b2ce5ea5dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:11.374180 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-7c9fd554-5abd-44c8-a65d-5b2ce5ea5dc0 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:11.376029 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-7c9fd554-5abd-44c8-a65d-5b2ce5ea5dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:11.385263 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1057/2100] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:11.391874 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9fd554-5abd-44c8-a65d-5b2ce5ea5dc0 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:11.392496 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1044/2101] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:11 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:11.404619 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9fd554-5abd-44c8-a65d-5b2ce5ea5dc0 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:11.412838 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e311f929-c911-40b5-ac09-8f13e60b2139 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:11.412838 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-e311f929-c911-40b5-ac09-8f13e60b2139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:11.414002 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb404b2-6dd6-4008-8532-27a6e030c3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:11.414702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb404b2-6dd6-4008-8532-27a6e030c3a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:11.415051 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb404b2-6dd6-4008-8532-27a6e030c3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:11.415389 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb404b2-6dd6-4008-8532-27a6e030c3a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:11.416124 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb404b2-6dd6-4008-8532-27a6e030c3a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:46:11.416786 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb404b2-6dd6-4008-8532-27a6e030c3a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:11.418914 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb404b2-6dd6-4008-8532-27a6e030c3a5 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:11.435285 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1058/2102] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:46:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:46:11.445698 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb404b2-6dd6-4008-8532-27a6e030c3a5 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:11.470064 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb404b2-6dd6-4008-8532-27a6e030c3a5 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:11.478922 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13baf711-951c-4c37-ac27-da8c5678c492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:11.479724 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13baf711-951c-4c37-ac27-da8c5678c492 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:11.480124 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13baf711-951c-4c37-ac27-da8c5678c492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:11.480522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13baf711-951c-4c37-ac27-da8c5678c492 admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:11.482225 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13baf711-951c-4c37-ac27-da8c5678c492 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:11.484274 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13baf711-951c-4c37-ac27-da8c5678c492 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:11.514031 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e311f929-c911-40b5-ac09-8f13e60b2139 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:11.518881 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301394bb-8e0a-4d9b-9076-f6e3f545e2f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:11.519526 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301394bb-8e0a-4d9b-9076-f6e3f545e2f7 service cinder] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:11.519834 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301394bb-8e0a-4d9b-9076-f6e3f545e2f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:11.520119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301394bb-8e0a-4d9b-9076-f6e3f545e2f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:11.531730 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301394bb-8e0a-4d9b-9076-f6e3f545e2f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:11.533444 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301394bb-8e0a-4d9b-9076-f6e3f545e2f7 service cinder] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:11.548119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1045/2103] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:11 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:11.579483 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1059/2104] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:11 2022] POST /identity/v3/projects => generated 373 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:11.584136 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301394bb-8e0a-4d9b-9076-f6e3f545e2f7 service cinder] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:11.594655 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301394bb-8e0a-4d9b-9076-f6e3f545e2f7 service cinder] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:11.660533 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ea7a9-3dc2-4ea8-a455-2cd5b0d591be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:11.660902 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ea7a9-3dc2-4ea8-a455-2cd5b0d591be admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:11.661228 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ea7a9-3dc2-4ea8-a455-2cd5b0d591be admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:11.661496 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ea7a9-3dc2-4ea8-a455-2cd5b0d591be admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:11.662099 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366ea7a9-3dc2-4ea8-a455-2cd5b0d591be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:11.664359 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366ea7a9-3dc2-4ea8-a455-2cd5b0d591be admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:11.681736 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:11.766953 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1046/2105] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:11 2022] POST /identity/v3/users => generated 287 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:11.770887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13baf711-951c-4c37-ac27-da8c5678c492 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:11.781395 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13baf711-951c-4c37-ac27-da8c5678c492 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:11.787281 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa31e6e7-5a5e-4733-a926-09e55d24b111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:11.787990 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa31e6e7-5a5e-4733-a926-09e55d24b111 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:11.788343 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa31e6e7-5a5e-4733-a926-09e55d24b111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:11.788728 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa31e6e7-5a5e-4733-a926-09e55d24b111 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:11.789461 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa31e6e7-5a5e-4733-a926-09e55d24b111 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:11.791081 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa31e6e7-5a5e-4733-a926-09e55d24b111 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:11.802960 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1060/2106] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:11.807427 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ea7a9-3dc2-4ea8-a455-2cd5b0d591be admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:11.818984 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ea7a9-3dc2-4ea8-a455-2cd5b0d591be admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:11.827835 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eec75f-af32-425d-a37a-74c780c71633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:11.827835 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eec75f-af32-425d-a37a-74c780c71633 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:11.827835 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eec75f-af32-425d-a37a-74c780c71633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:11.827835 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eec75f-af32-425d-a37a-74c780c71633 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:11.828505 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eec75f-af32-425d-a37a-74c780c71633 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:11.829101 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eec75f-af32-425d-a37a-74c780c71633 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:11.843211 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa31e6e7-5a5e-4733-a926-09e55d24b111 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:11.846843 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1047/2107] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:11.849012 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-fa31e6e7-5a5e-4733-a926-09e55d24b111 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:11.851168 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eec75f-af32-425d-a37a-74c780c71633 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:11.864286 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eec75f-af32-425d-a37a-74c780c71633 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:11.873668 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fdeea3-e7aa-4a8f-ab97-d45b7e55258b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:11.874519 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fdeea3-e7aa-4a8f-ab97-d45b7e55258b admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:11.874794 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fdeea3-e7aa-4a8f-ab97-d45b7e55258b admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:11.875248 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fdeea3-e7aa-4a8f-ab97-d45b7e55258b admin admin] PATH_INFO: `/v3/projects/6738e05eed4849e5b67d8b8d30f22269/users/be903033452a42eeb3905b5dc214945d/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:11.895336 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fdeea3-e7aa-4a8f-ab97-d45b7e55258b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6738e05eed4849e5b67d8b8d30f22269, user_id=be903033452a42eeb3905b5dc214945d, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:11.898182 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fdeea3-e7aa-4a8f-ab97-d45b7e55258b admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:11.929339 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa31e6e7-5a5e-4733-a926-09e55d24b111 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:11.932257 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:11.935359 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c811ce4-8d81-4795-9fa6-fa51a7cfb1fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:11.935982 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c811ce4-8d81-4795-9fa6-fa51a7cfb1fb service cinder] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:11.936688 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c811ce4-8d81-4795-9fa6-fa51a7cfb1fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:11.936688 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c811ce4-8d81-4795-9fa6-fa51a7cfb1fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:11.938508 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-3c811ce4-8d81-4795-9fa6-fa51a7cfb1fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:12.013397 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c811ce4-8d81-4795-9fa6-fa51a7cfb1fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:12.015134 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c811ce4-8d81-4795-9fa6-fa51a7cfb1fb service cinder] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:12.026054 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1061/2108] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:46:12.058737 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1048/2109] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:11 2022] PUT /identity/v3/projects/6738e05eed4849e5b67d8b8d30f22269/users/be903033452a42eeb3905b5dc214945d/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:46:12.072304 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d40a6-5d51-4789-9666-eb7a1caae6ca None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:12.073106 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d40a6-5d51-4789-9666-eb7a1caae6ca None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:12.073237 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d40a6-5d51-4789-9666-eb7a1caae6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:12.119213 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-fc1d40a6-5d51-4789-9666-eb7a1caae6ca None None] MFA Rules not processed for user `be903033452a42eeb3905b5dc214945d`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:12.120593 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-fc1d40a6-5d51-4789-9666-eb7a1caae6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:12.205852 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1062/2110] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:46:12.217204 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fdeea3-e7aa-4a8f-ab97-d45b7e55258b admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:12.282700 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fdeea3-e7aa-4a8f-ab97-d45b7e55258b admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:12.287972 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a644a-179d-4241-8cb6-e1270d576894 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:12.288628 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a644a-179d-4241-8cb6-e1270d576894 service cinder] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:12.288917 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a644a-179d-4241-8cb6-e1270d576894 service cinder] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:12.289265 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a644a-179d-4241-8cb6-e1270d576894 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:12.298382 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80a644a-179d-4241-8cb6-e1270d576894 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:12.300560 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80a644a-179d-4241-8cb6-e1270d576894 service cinder] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:12.310376 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1049/2111] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:13.004906 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:13.730859 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:15.374321 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:16.670160 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e7d52-58b0-4b4f-8a16-8162c072045b None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:16.670698 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e7d52-58b0-4b4f-8a16-8162c072045b None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:16.670776 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e7d52-58b0-4b4f-8a16-8162c072045b None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:16.673000 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1063/2112] 10.222.0.15 () {54 vars in 921 bytes} [Wed Aug 24 05:46:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:46:16.682662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca3ff8-4601-422c-b194-84301a05443f None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:16.683202 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca3ff8-4601-422c-b194-84301a05443f None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:16.683808 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca3ff8-4601-422c-b194-84301a05443f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:16.712738 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-5bca3ff8-4601-422c-b194-84301a05443f None None] MFA Rules not processed for user `6778b7c5ad19484b877494f143050405`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:16.714365 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-5bca3ff8-4601-422c-b194-84301a05443f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:16.795036 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1050/2113] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Aug 24 05:46:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:16.806943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e7d52-58b0-4b4f-8a16-8162c072045b None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:16.808739 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-1f6e7d52-58b0-4b4f-8a16-8162c072045b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:16.828000 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e7d52-58b0-4b4f-8a16-8162c072045b None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:16.833573 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1df55f-3c04-43bb-b542-809d1734042e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6778b7c5ad19484b877494f143050405', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:16.834195 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1df55f-3c04-43bb-b542-809d1734042e service placement] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:16.834481 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1df55f-3c04-43bb-b542-809d1734042e service placement] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:16.834933 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1df55f-3c04-43bb-b542-809d1734042e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:16.844611 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1df55f-3c04-43bb-b542-809d1734042e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:16.846477 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1df55f-3c04-43bb-b542-809d1734042e service placement] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:16.857008 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1064/2114] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:16.893774 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca3ff8-4601-422c-b194-84301a05443f None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:16.895400 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-5bca3ff8-4601-422c-b194-84301a05443f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:16.983195 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca3ff8-4601-422c-b194-84301a05443f None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:16.988827 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47b041-c8bb-47ee-a1b0-5397440a4fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:16.989507 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47b041-c8bb-47ee-a1b0-5397440a4fac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:16.989791 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47b041-c8bb-47ee-a1b0-5397440a4fac admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:16.992017 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47b041-c8bb-47ee-a1b0-5397440a4fac admin admin] PATH_INFO: `/v3/users/9bda9f6f00c74112a54573c3905ad1bf` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:16.992017 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a47b041-c8bb-47ee-a1b0-5397440a4fac admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bda9f6f00c74112a54573c3905ad1bf)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:16.993179 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a47b041-c8bb-47ee-a1b0-5397440a4fac admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:17.217256 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-7a47b041-c8bb-47ee-a1b0-5397440a4fac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bda9f6f00c74112a54573c3905ad1bf'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:17.227170 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-7a47b041-c8bb-47ee-a1b0-5397440a4fac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bda9f6f00c74112a54573c3905ad1bf'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:17.228125 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-7a47b041-c8bb-47ee-a1b0-5397440a4fac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bda9f6f00c74112a54573c3905ad1bf'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:17.336641 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-7a47b041-c8bb-47ee-a1b0-5397440a4fac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bda9f6f00c74112a54573c3905ad1bf'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:17.350467 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-7a47b041-c8bb-47ee-a1b0-5397440a4fac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bda9f6f00c74112a54573c3905ad1bf'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:17.357273 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1051/2115] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:16 2022] DELETE /identity/v3/users/9bda9f6f00c74112a54573c3905ad1bf => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:17.440775 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:17.595000 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:17.710934 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1df55f-3c04-43bb-b542-809d1734042e service placement] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:17.729518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1df55f-3c04-43bb-b542-809d1734042e service placement] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:17.796830 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af4207-d5da-4c2d-8595-aea4d83ea6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:17.797496 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af4207-d5da-4c2d-8595-aea4d83ea6e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:17.797783 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af4207-d5da-4c2d-8595-aea4d83ea6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:17.798062 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af4207-d5da-4c2d-8595-aea4d83ea6e5 admin admin] PATH_INFO: `/v3/projects/eb377db175354cac93e5ff6579af3e3e` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:17.799842 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af4207-d5da-4c2d-8595-aea4d83ea6e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb377db175354cac93e5ff6579af3e3e)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:17.801891 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af4207-d5da-4c2d-8595-aea4d83ea6e5 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:17.945695 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-17af4207-d5da-4c2d-8595-aea4d83ea6e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb377db175354cac93e5ff6579af3e3e', 'request_id': 'req-17af4207-d5da-4c2d-8595-aea4d83ea6e5'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:17.951985 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-17af4207-d5da-4c2d-8595-aea4d83ea6e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb377db175354cac93e5ff6579af3e3e', 'request_id': 'req-17af4207-d5da-4c2d-8595-aea4d83ea6e5'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:18.062864 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-17af4207-d5da-4c2d-8595-aea4d83ea6e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb377db175354cac93e5ff6579af3e3e', 'request_id': 'req-17af4207-d5da-4c2d-8595-aea4d83ea6e5'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:18.081396 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-17af4207-d5da-4c2d-8595-aea4d83ea6e5 admin admin] The token cache is being invalidate because project eb377db175354cac93e5ff6579af3e3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:18.081720 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-17af4207-d5da-4c2d-8595-aea4d83ea6e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:18.082386 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-17af4207-d5da-4c2d-8595-aea4d83ea6e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:18.083910 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1065/2116] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:17 2022] DELETE /identity/v3/projects/eb377db175354cac93e5ff6579af3e3e => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:46:18.089569 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f8aeb3-6e0e-46a0-821f-a699dd2bef9b None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:18.090210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f8aeb3-6e0e-46a0-821f-a699dd2bef9b None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:18.090210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f8aeb3-6e0e-46a0-821f-a699dd2bef9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:18.122202 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-29f8aeb3-6e0e-46a0-821f-a699dd2bef9b None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:18.123564 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-29f8aeb3-6e0e-46a0-821f-a699dd2bef9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:18.193141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1052/2117] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:18.201264 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0652f-4b3c-44ce-ad84-3fa70979d1ce None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:18.201483 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0652f-4b3c-44ce-ad84-3fa70979d1ce None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:18.201767 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0652f-4b3c-44ce-ad84-3fa70979d1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:18.222921 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-19a0652f-4b3c-44ce-ad84-3fa70979d1ce None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:18.224618 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-19a0652f-4b3c-44ce-ad84-3fa70979d1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:18.232609 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1066/2118] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:18.236825 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f8aeb3-6e0e-46a0-821f-a699dd2bef9b None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:18.249373 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f8aeb3-6e0e-46a0-821f-a699dd2bef9b None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:18.257094 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a4d51d-fe60-4b40-bf09-868db5357333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:18.257721 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a4d51d-fe60-4b40-bf09-868db5357333 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:18.258001 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a4d51d-fe60-4b40-bf09-868db5357333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:18.258273 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a4d51d-fe60-4b40-bf09-868db5357333 admin admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:18.258926 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a4d51d-fe60-4b40-bf09-868db5357333 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:46:18.259306 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a4d51d-fe60-4b40-bf09-868db5357333 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:18.261057 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a4d51d-fe60-4b40-bf09-868db5357333 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:18.274813 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1053/2119] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:46:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:46:18.279257 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0652f-4b3c-44ce-ad84-3fa70979d1ce None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:18.290752 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0652f-4b3c-44ce-ad84-3fa70979d1ce None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:18.301777 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545c62a5-c50f-4417-a8e5-62a3cf67504e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:18.301777 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c62a5-c50f-4417-a8e5-62a3cf67504e admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:18.302922 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c62a5-c50f-4417-a8e5-62a3cf67504e admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:18.302922 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c62a5-c50f-4417-a8e5-62a3cf67504e admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:18.302922 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545c62a5-c50f-4417-a8e5-62a3cf67504e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:18.306285 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545c62a5-c50f-4417-a8e5-62a3cf67504e admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:18.454023 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1067/2120] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:18 2022] POST /identity/v3/projects => generated 387 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:46:18.456144 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a4d51d-fe60-4b40-bf09-868db5357333 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:18.466378 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a4d51d-fe60-4b40-bf09-868db5357333 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:18.528257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c71c2b-463e-4d8c-bc79-ea3e8252a817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:18.533968 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c71c2b-463e-4d8c-bc79-ea3e8252a817 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:18.533968 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c71c2b-463e-4d8c-bc79-ea3e8252a817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:18.533968 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c71c2b-463e-4d8c-bc79-ea3e8252a817 admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:18.533968 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c71c2b-463e-4d8c-bc79-ea3e8252a817 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:18.533968 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c71c2b-463e-4d8c-bc79-ea3e8252a817 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:18.653350 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1054/2121] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:18 2022] POST /identity/v3/users => generated 294 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:18.653525 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545c62a5-c50f-4417-a8e5-62a3cf67504e admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:18.658943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545c62a5-c50f-4417-a8e5-62a3cf67504e admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:18.664677 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b2498-054d-45a4-bb7f-5c48e7204ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:18.665272 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b2498-054d-45a4-bb7f-5c48e7204ba5 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:18.665548 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b2498-054d-45a4-bb7f-5c48e7204ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:18.665817 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b2498-054d-45a4-bb7f-5c48e7204ba5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:18.666728 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425b2498-054d-45a4-bb7f-5c48e7204ba5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:18.668487 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425b2498-054d-45a4-bb7f-5c48e7204ba5 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:18.682655 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1068/2122] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:18.688866 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c71c2b-463e-4d8c-bc79-ea3e8252a817 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:18.700845 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c71c2b-463e-4d8c-bc79-ea3e8252a817 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:18.707631 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba55a4-b908-4b3b-a436-059a9f5e8cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:18.707948 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba55a4-b908-4b3b-a436-059a9f5e8cf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:18.708234 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba55a4-b908-4b3b-a436-059a9f5e8cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:18.708535 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba55a4-b908-4b3b-a436-059a9f5e8cf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:18.709249 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba55a4-b908-4b3b-a436-059a9f5e8cf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:18.710885 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba55a4-b908-4b3b-a436-059a9f5e8cf4 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:18.723128 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1055/2123] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:18.727470 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b2498-054d-45a4-bb7f-5c48e7204ba5 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:18.738622 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b2498-054d-45a4-bb7f-5c48e7204ba5 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:18.743350 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd95f5e-f15c-4f2e-886e-f5d1a06a3c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:18.744008 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd95f5e-f15c-4f2e-886e-f5d1a06a3c89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:18.744289 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd95f5e-f15c-4f2e-886e-f5d1a06a3c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:18.744598 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd95f5e-f15c-4f2e-886e-f5d1a06a3c89 admin admin] PATH_INFO: `/v3/projects/56fbd96812654d16bbcf3522574a67eb/users/c7ef3bf8dfd342f5a2fef02bf597af90/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:18.748926 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:18.763750 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd95f5e-f15c-4f2e-886e-f5d1a06a3c89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56fbd96812654d16bbcf3522574a67eb, user_id=c7ef3bf8dfd342f5a2fef02bf597af90, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:18.766505 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd95f5e-f15c-4f2e-886e-f5d1a06a3c89 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:18.839850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1069/2124] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:18 2022] PUT /identity/v3/projects/56fbd96812654d16bbcf3522574a67eb/users/c7ef3bf8dfd342f5a2fef02bf597af90/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:18.845043 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9055bf-1bc0-49c2-a00a-6e5f606a112d None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:18.845327 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9055bf-1bc0-49c2-a00a-6e5f606a112d None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:18.845632 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9055bf-1bc0-49c2-a00a-6e5f606a112d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:18.866044 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-9d9055bf-1bc0-49c2-a00a-6e5f606a112d None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:18.867387 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-9d9055bf-1bc0-49c2-a00a-6e5f606a112d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:18.935449 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1056/2125] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:18.945723 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd95f5e-f15c-4f2e-886e-f5d1a06a3c89 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:18.947642 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-8cd95f5e-f15c-4f2e-886e-f5d1a06a3c89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:19.007759 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd95f5e-f15c-4f2e-886e-f5d1a06a3c89 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:19.011593 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db5b49-9a6e-4362-b834-95ef8bd85074 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:19.012240 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db5b49-9a6e-4362-b834-95ef8bd85074 service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:19.012539 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db5b49-9a6e-4362-b834-95ef8bd85074 service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:19.012818 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db5b49-9a6e-4362-b834-95ef8bd85074 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:19.023028 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32db5b49-9a6e-4362-b834-95ef8bd85074 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:19.024493 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32db5b49-9a6e-4362-b834-95ef8bd85074 service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:19.034406 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1070/2126] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:19.120034 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:19.794625 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:20.098667 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:20.328056 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:21.108878 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:25.458848 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2a35a-ffcd-4af5-9f3d-a8292f326e33 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:25.458848 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2a35a-ffcd-4af5-9f3d-a8292f326e33 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:25.459532 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2a35a-ffcd-4af5-9f3d-a8292f326e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:25.492374 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-73b2a35a-ffcd-4af5-9f3d-a8292f326e33 None None] MFA Rules not processed for user `c7ef3bf8dfd342f5a2fef02bf597af90`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:25.493921 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-73b2a35a-ffcd-4af5-9f3d-a8292f326e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:25.518083 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:25.619016 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1057/2127] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:25 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:25.627935 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db5b49-9a6e-4362-b834-95ef8bd85074 service neutron] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:25.632272 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-32db5b49-9a6e-4362-b834-95ef8bd85074 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:25.780176 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db5b49-9a6e-4362-b834-95ef8bd85074 service neutron] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:25.790839 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d28cb8-061d-4fca-9f7d-01f44d9fbabb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:25.790839 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d28cb8-061d-4fca-9f7d-01f44d9fbabb service nova] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:25.790839 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d28cb8-061d-4fca-9f7d-01f44d9fbabb service nova] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:25.790839 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d28cb8-061d-4fca-9f7d-01f44d9fbabb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:25.812565 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d28cb8-061d-4fca-9f7d-01f44d9fbabb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:25.814360 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d28cb8-061d-4fca-9f7d-01f44d9fbabb service nova] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:25.829520 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1071/2128] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:25 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:26.073772 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2a35a-ffcd-4af5-9f3d-a8292f326e33 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:26.075383 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-73b2a35a-ffcd-4af5-9f3d-a8292f326e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:26.161284 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2a35a-ffcd-4af5-9f3d-a8292f326e33 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:26.173254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5009c8fb-034a-4b30-82c9-2015a18bcbd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:26.173961 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5009c8fb-034a-4b30-82c9-2015a18bcbd9 service cinder] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:26.174359 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5009c8fb-034a-4b30-82c9-2015a18bcbd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:26.174695 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5009c8fb-034a-4b30-82c9-2015a18bcbd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:26.180947 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-5009c8fb-034a-4b30-82c9-2015a18bcbd9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:26.283074 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5009c8fb-034a-4b30-82c9-2015a18bcbd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:26.286269 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5009c8fb-034a-4b30-82c9-2015a18bcbd9 service cinder] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:26.297501 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1058/2129] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:26 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:29.523190 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabf1d0-71ed-45d5-9b6c-db37f25bea96 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:29.524090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabf1d0-71ed-45d5-9b6c-db37f25bea96 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:29.526083 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabf1d0-71ed-45d5-9b6c-db37f25bea96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:29.560496 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-edabf1d0-71ed-45d5-9b6c-db37f25bea96 None None] MFA Rules not processed for user `c7ef3bf8dfd342f5a2fef02bf597af90`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:29.561778 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-edabf1d0-71ed-45d5-9b6c-db37f25bea96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:29.575084 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1072/2130] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:29 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:31.647235 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5009c8fb-034a-4b30-82c9-2015a18bcbd9 service cinder] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:31.649043 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-5009c8fb-034a-4b30-82c9-2015a18bcbd9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:31.675204 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5009c8fb-034a-4b30-82c9-2015a18bcbd9 service cinder] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:31.681757 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f872c-26a3-4d1f-969d-455364d5bf3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:31.682940 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f872c-26a3-4d1f-969d-455364d5bf3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:31.683471 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f872c-26a3-4d1f-969d-455364d5bf3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:31.683944 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f872c-26a3-4d1f-969d-455364d5bf3a admin admin] PATH_INFO: `/v3/users/77cb75e0dea3422b9d31e740636c5641` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:31.686435 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4f872c-26a3-4d1f-969d-455364d5bf3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=77cb75e0dea3422b9d31e740636c5641)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:31.690650 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4f872c-26a3-4d1f-969d-455364d5bf3a admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:31.947525 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-7f4f872c-26a3-4d1f-969d-455364d5bf3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77cb75e0dea3422b9d31e740636c5641'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:31.956000 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-7f4f872c-26a3-4d1f-969d-455364d5bf3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77cb75e0dea3422b9d31e740636c5641'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:31.956808 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-7f4f872c-26a3-4d1f-969d-455364d5bf3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77cb75e0dea3422b9d31e740636c5641'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:32.017870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-7f4f872c-26a3-4d1f-969d-455364d5bf3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77cb75e0dea3422b9d31e740636c5641'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:32.021686 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:32.051678 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-7f4f872c-26a3-4d1f-969d-455364d5bf3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77cb75e0dea3422b9d31e740636c5641'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:32.081310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1059/2131] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:31 2022] DELETE /identity/v3/users/77cb75e0dea3422b9d31e740636c5641 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:32.088321 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691c2f2-3669-4872-a3d8-508839d3864c None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:32.089179 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691c2f2-3669-4872-a3d8-508839d3864c None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:32.089315 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691c2f2-3669-4872-a3d8-508839d3864c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:32.144697 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-4691c2f2-3669-4872-a3d8-508839d3864c None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:32.147357 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-4691c2f2-3669-4872-a3d8-508839d3864c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:32.258495 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1073/2132] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:32.269352 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f872c-26a3-4d1f-969d-455364d5bf3a admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:32.271528 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-7f4f872c-26a3-4d1f-969d-455364d5bf3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:32.368481 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f872c-26a3-4d1f-969d-455364d5bf3a admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:32.373579 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9eb5ad-d273-4e7b-ab03-546049f4c245 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:32.374352 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9eb5ad-d273-4e7b-ab03-546049f4c245 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:32.374532 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9eb5ad-d273-4e7b-ab03-546049f4c245 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:32.374849 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9eb5ad-d273-4e7b-ab03-546049f4c245 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:32.388284 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9eb5ad-d273-4e7b-ab03-546049f4c245 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:32.389220 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9eb5ad-d273-4e7b-ab03-546049f4c245 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:32.401111 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1060/2133] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:33.088886 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4691c2f2-3669-4872-a3d8-508839d3864c None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:33.091202 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-4691c2f2-3669-4872-a3d8-508839d3864c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:33.114500 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4691c2f2-3669-4872-a3d8-508839d3864c None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:33.122879 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eba9d67-88ba-4e13-b380-6a1bc0e14f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:33.124052 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba9d67-88ba-4e13-b380-6a1bc0e14f8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:33.124825 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba9d67-88ba-4e13-b380-6a1bc0e14f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:33.125398 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba9d67-88ba-4e13-b380-6a1bc0e14f8a admin admin] PATH_INFO: `/v3/projects/ec7531f7ea474d88a50104ea32362867` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:33.127747 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eba9d67-88ba-4e13-b380-6a1bc0e14f8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec7531f7ea474d88a50104ea32362867)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:33.132548 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eba9d67-88ba-4e13-b380-6a1bc0e14f8a admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:33.290688 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-6eba9d67-88ba-4e13-b380-6a1bc0e14f8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec7531f7ea474d88a50104ea32362867', 'request_id': 'req-6eba9d67-88ba-4e13-b380-6a1bc0e14f8a'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:33.298189 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-6eba9d67-88ba-4e13-b380-6a1bc0e14f8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec7531f7ea474d88a50104ea32362867', 'request_id': 'req-6eba9d67-88ba-4e13-b380-6a1bc0e14f8a'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:33.377573 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-6eba9d67-88ba-4e13-b380-6a1bc0e14f8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec7531f7ea474d88a50104ea32362867', 'request_id': 'req-6eba9d67-88ba-4e13-b380-6a1bc0e14f8a'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:33.382354 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:33.389866 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-6eba9d67-88ba-4e13-b380-6a1bc0e14f8a admin admin] The token cache is being invalidate because project ec7531f7ea474d88a50104ea32362867 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:33.390160 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-6eba9d67-88ba-4e13-b380-6a1bc0e14f8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:33.390842 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-6eba9d67-88ba-4e13-b380-6a1bc0e14f8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:33.394782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1074/2134] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:33 2022] DELETE /identity/v3/projects/ec7531f7ea474d88a50104ea32362867 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:33.399321 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9eb5ad-d273-4e7b-ab03-546049f4c245 service neutron] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:33.410869 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9eb5ad-d273-4e7b-ab03-546049f4c245 service neutron] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:33.476461 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f93eaa-e026-4251-844a-2d6bafe3a1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:33.477526 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f93eaa-e026-4251-844a-2d6bafe3a1ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:33.477526 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f93eaa-e026-4251-844a-2d6bafe3a1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:33.477978 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f93eaa-e026-4251-844a-2d6bafe3a1ca admin admin] PATH_INFO: `/v3/users/04f89623c2cd45f6865af11229eb38fa` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:33.479166 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f93eaa-e026-4251-844a-2d6bafe3a1ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=04f89623c2cd45f6865af11229eb38fa)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:33.481020 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f93eaa-e026-4251-844a-2d6bafe3a1ca admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:33.760386 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-59f93eaa-e026-4251-844a-2d6bafe3a1ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04f89623c2cd45f6865af11229eb38fa'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:33.769261 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-59f93eaa-e026-4251-844a-2d6bafe3a1ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04f89623c2cd45f6865af11229eb38fa'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:33.770056 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-59f93eaa-e026-4251-844a-2d6bafe3a1ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04f89623c2cd45f6865af11229eb38fa'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:33.863560 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-59f93eaa-e026-4251-844a-2d6bafe3a1ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04f89623c2cd45f6865af11229eb38fa'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:33.883617 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-59f93eaa-e026-4251-844a-2d6bafe3a1ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04f89623c2cd45f6865af11229eb38fa'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:33.893139 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1061/2135] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:33 2022] DELETE /identity/v3/users/04f89623c2cd45f6865af11229eb38fa => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:34.416348 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eba9d67-88ba-4e13-b380-6a1bc0e14f8a admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:34.418409 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-6eba9d67-88ba-4e13-b380-6a1bc0e14f8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:34.434058 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eba9d67-88ba-4e13-b380-6a1bc0e14f8a admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:34.510275 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb9946-1525-4d65-a464-bcfaa5702628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:34.510967 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb9946-1525-4d65-a464-bcfaa5702628 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:34.511247 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb9946-1525-4d65-a464-bcfaa5702628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:34.511515 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb9946-1525-4d65-a464-bcfaa5702628 admin admin] PATH_INFO: `/v3/projects/1afe0bbede2a4abebea9cfa7eedbb5ff` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:34.512825 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fb9946-1525-4d65-a464-bcfaa5702628 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1afe0bbede2a4abebea9cfa7eedbb5ff)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:34.514758 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fb9946-1525-4d65-a464-bcfaa5702628 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:34.669032 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-12fb9946-1525-4d65-a464-bcfaa5702628 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1afe0bbede2a4abebea9cfa7eedbb5ff', 'request_id': 'req-12fb9946-1525-4d65-a464-bcfaa5702628'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:34.675239 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-12fb9946-1525-4d65-a464-bcfaa5702628 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1afe0bbede2a4abebea9cfa7eedbb5ff', 'request_id': 'req-12fb9946-1525-4d65-a464-bcfaa5702628'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:34.759818 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-12fb9946-1525-4d65-a464-bcfaa5702628 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1afe0bbede2a4abebea9cfa7eedbb5ff', 'request_id': 'req-12fb9946-1525-4d65-a464-bcfaa5702628'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:34.773968 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-12fb9946-1525-4d65-a464-bcfaa5702628 admin admin] The token cache is being invalidate because project 1afe0bbede2a4abebea9cfa7eedbb5ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:34.774338 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-12fb9946-1525-4d65-a464-bcfaa5702628 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:34.774987 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-12fb9946-1525-4d65-a464-bcfaa5702628 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:34.776810 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1075/2136] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:34 2022] DELETE /identity/v3/projects/1afe0bbede2a4abebea9cfa7eedbb5ff => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:34.782805 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5871a227-1304-4f0f-9584-ac6de2f96470 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:34.783360 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5871a227-1304-4f0f-9584-ac6de2f96470 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:34.783548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5871a227-1304-4f0f-9584-ac6de2f96470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:34.793946 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:34.808624 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-5871a227-1304-4f0f-9584-ac6de2f96470 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:34.810831 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-5871a227-1304-4f0f-9584-ac6de2f96470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:34.879608 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1062/2137] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:34.887519 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469981f-df53-47e2-ac88-87d9b0c12678 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:34.890516 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469981f-df53-47e2-ac88-87d9b0c12678 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:34.890516 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469981f-df53-47e2-ac88-87d9b0c12678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:34.912046 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-f469981f-df53-47e2-ac88-87d9b0c12678 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:34.913447 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-f469981f-df53-47e2-ac88-87d9b0c12678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:34.927462 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1076/2138] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:34.928822 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5871a227-1304-4f0f-9584-ac6de2f96470 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:34.940107 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5871a227-1304-4f0f-9584-ac6de2f96470 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:34.945459 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017dce4-167c-48ee-8dda-9e98263af514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:34.946943 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017dce4-167c-48ee-8dda-9e98263af514 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:34.947304 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017dce4-167c-48ee-8dda-9e98263af514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:34.947654 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017dce4-167c-48ee-8dda-9e98263af514 admin admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:34.948316 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a017dce4-167c-48ee-8dda-9e98263af514 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:46:34.948668 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a017dce4-167c-48ee-8dda-9e98263af514 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:34.950434 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a017dce4-167c-48ee-8dda-9e98263af514 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:34.971846 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1063/2139] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:46:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:46:34.977858 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f469981f-df53-47e2-ac88-87d9b0c12678 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:34.988592 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f469981f-df53-47e2-ac88-87d9b0c12678 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:34.995392 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b394f-2fb0-481c-a8e2-3df65eb189a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:34.996361 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b394f-2fb0-481c-a8e2-3df65eb189a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:34.996826 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b394f-2fb0-481c-a8e2-3df65eb189a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:34.997404 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b394f-2fb0-481c-a8e2-3df65eb189a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:34.998223 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b394f-2fb0-481c-a8e2-3df65eb189a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:35.000301 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b394f-2fb0-481c-a8e2-3df65eb189a0 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:35.102646 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1077/2140] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:34 2022] POST /identity/v3/projects => generated 369 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:35.107099 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017dce4-167c-48ee-8dda-9e98263af514 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:35.118866 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017dce4-167c-48ee-8dda-9e98263af514 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:35.181996 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d50c1b-0f6e-4811-a94d-2e37f5427302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:35.182980 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d50c1b-0f6e-4811-a94d-2e37f5427302 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:35.183264 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d50c1b-0f6e-4811-a94d-2e37f5427302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:35.183536 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d50c1b-0f6e-4811-a94d-2e37f5427302 admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:35.184499 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d50c1b-0f6e-4811-a94d-2e37f5427302 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:35.186927 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d50c1b-0f6e-4811-a94d-2e37f5427302 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:35.290410 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1064/2141] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:35 2022] POST /identity/v3/users => generated 285 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:35.295279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b394f-2fb0-481c-a8e2-3df65eb189a0 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:35.307454 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b394f-2fb0-481c-a8e2-3df65eb189a0 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:35.312509 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3dc39a-accb-462e-a978-926383ada41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:35.313128 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3dc39a-accb-462e-a978-926383ada41e admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:35.313409 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3dc39a-accb-462e-a978-926383ada41e admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:35.313689 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3dc39a-accb-462e-a978-926383ada41e admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:35.314396 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3dc39a-accb-462e-a978-926383ada41e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:35.316550 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3dc39a-accb-462e-a978-926383ada41e admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:35.327756 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1078/2142] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:35.332802 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d50c1b-0f6e-4811-a94d-2e37f5427302 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:35.346736 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d50c1b-0f6e-4811-a94d-2e37f5427302 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:35.351799 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bcd968-f816-400a-ace9-5fa60f770825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:35.352438 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bcd968-f816-400a-ace9-5fa60f770825 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:35.352724 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bcd968-f816-400a-ace9-5fa60f770825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:35.352996 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bcd968-f816-400a-ace9-5fa60f770825 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:35.353641 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bcd968-f816-400a-ace9-5fa60f770825 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:35.355242 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bcd968-f816-400a-ace9-5fa60f770825 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:35.368145 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1065/2143] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:35.372437 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3dc39a-accb-462e-a978-926383ada41e admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:35.383289 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3dc39a-accb-462e-a978-926383ada41e admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:35.387953 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f527ef-b588-44e7-9267-f447962d4457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:35.388674 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f527ef-b588-44e7-9267-f447962d4457 admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:35.388953 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f527ef-b588-44e7-9267-f447962d4457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:35.389240 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f527ef-b588-44e7-9267-f447962d4457 admin admin] PATH_INFO: `/v3/projects/6c07362c048745f5a9c69b821258d558/users/b5c56c037b3740c9b79456d755145841/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:35.404950 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f527ef-b588-44e7-9267-f447962d4457 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c07362c048745f5a9c69b821258d558, user_id=b5c56c037b3740c9b79456d755145841, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:35.407570 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f527ef-b588-44e7-9267-f447962d4457 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:35.487878 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1079/2144] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:35 2022] PUT /identity/v3/projects/6c07362c048745f5a9c69b821258d558/users/b5c56c037b3740c9b79456d755145841/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:35.504084 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f3aed-af43-4dbe-b1c9-f73d8d332dfc None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:35.504379 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f3aed-af43-4dbe-b1c9-f73d8d332dfc None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:35.504704 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f3aed-af43-4dbe-b1c9-f73d8d332dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:35.554972 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-c50f3aed-af43-4dbe-b1c9-f73d8d332dfc None None] MFA Rules not processed for user `b5c56c037b3740c9b79456d755145841`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:35.556291 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-c50f3aed-af43-4dbe-b1c9-f73d8d332dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:35.625351 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:35.633834 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1066/2145] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:35.638627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f527ef-b588-44e7-9267-f447962d4457 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:35.666307 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:35.706105 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f527ef-b588-44e7-9267-f447962d4457 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:35.710958 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ddc559-6cad-4c4d-a44e-9cae82556170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:35.711579 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ddc559-6cad-4c4d-a44e-9cae82556170 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:35.711861 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ddc559-6cad-4c4d-a44e-9cae82556170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:35.712140 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ddc559-6cad-4c4d-a44e-9cae82556170 admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:35.712834 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ddc559-6cad-4c4d-a44e-9cae82556170 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:35.714769 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ddc559-6cad-4c4d-a44e-9cae82556170 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:35.836049 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1080/2146] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:35 2022] POST /identity/v3/projects => generated 367 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:35.837118 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f3aed-af43-4dbe-b1c9-f73d8d332dfc None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:35.851111 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f3aed-af43-4dbe-b1c9-f73d8d332dfc None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:35.946097 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e56fe6-6341-4639-8c50-d279810b5b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:35.947063 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e56fe6-6341-4639-8c50-d279810b5b48 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:35.947538 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e56fe6-6341-4639-8c50-d279810b5b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:35.948058 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e56fe6-6341-4639-8c50-d279810b5b48 admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:35.949349 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e56fe6-6341-4639-8c50-d279810b5b48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:35.953204 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e56fe6-6341-4639-8c50-d279810b5b48 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:36.091939 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1067/2147] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:35 2022] POST /identity/v3/users => generated 290 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:46:36.097115 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ddc559-6cad-4c4d-a44e-9cae82556170 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:36.108804 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ddc559-6cad-4c4d-a44e-9cae82556170 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:36.114832 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab8e09-5ed0-407e-bdfa-291ea6f11c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:36.115465 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab8e09-5ed0-407e-bdfa-291ea6f11c4d admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:36.115752 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab8e09-5ed0-407e-bdfa-291ea6f11c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:36.116132 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab8e09-5ed0-407e-bdfa-291ea6f11c4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:36.116873 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab8e09-5ed0-407e-bdfa-291ea6f11c4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:36.118522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab8e09-5ed0-407e-bdfa-291ea6f11c4d admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:36.131946 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1081/2148] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:46:36.136430 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e56fe6-6341-4639-8c50-d279810b5b48 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:36.153409 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e56fe6-6341-4639-8c50-d279810b5b48 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:36.159584 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd1795f-76d2-4f16-b0b5-17d21c046383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:36.160149 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1795f-76d2-4f16-b0b5-17d21c046383 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:36.160471 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1795f-76d2-4f16-b0b5-17d21c046383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:36.160754 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1795f-76d2-4f16-b0b5-17d21c046383 admin admin] PATH_INFO: `/v3/projects/5cc7e6d82b1341fab2008f5d338efb16/users/4114064accaa45ce9e993cffed949114/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:36.179112 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd1795f-76d2-4f16-b0b5-17d21c046383 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cc7e6d82b1341fab2008f5d338efb16, user_id=4114064accaa45ce9e993cffed949114, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:36.181784 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd1795f-76d2-4f16-b0b5-17d21c046383 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:36.252770 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1068/2149] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:36 2022] PUT /identity/v3/projects/5cc7e6d82b1341fab2008f5d338efb16/users/4114064accaa45ce9e993cffed949114/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:36.256874 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab8e09-5ed0-407e-bdfa-291ea6f11c4d admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:36.268008 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab8e09-5ed0-407e-bdfa-291ea6f11c4d admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:36.329450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a249acb7-51a7-44b3-a4ae-60d667ec6e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:36.330198 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a249acb7-51a7-44b3-a4ae-60d667ec6e45 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:36.330710 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a249acb7-51a7-44b3-a4ae-60d667ec6e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:36.331039 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a249acb7-51a7-44b3-a4ae-60d667ec6e45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:36.331704 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a249acb7-51a7-44b3-a4ae-60d667ec6e45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:36.333352 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a249acb7-51a7-44b3-a4ae-60d667ec6e45 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:36.344256 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1082/2150] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:36 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:36.349278 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd1795f-76d2-4f16-b0b5-17d21c046383 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:36.360146 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd1795f-76d2-4f16-b0b5-17d21c046383 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:36.366183 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e103006-ac9b-4901-97a1-47a0e0ee5320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:36.366854 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e103006-ac9b-4901-97a1-47a0e0ee5320 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:36.367142 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e103006-ac9b-4901-97a1-47a0e0ee5320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:36.367423 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e103006-ac9b-4901-97a1-47a0e0ee5320 admin admin] PATH_INFO: `/v3/projects/5cc7e6d82b1341fab2008f5d338efb16/users/4114064accaa45ce9e993cffed949114/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:36.369398 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e103006-ac9b-4901-97a1-47a0e0ee5320 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cc7e6d82b1341fab2008f5d338efb16, user_id=4114064accaa45ce9e993cffed949114, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:36.372110 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e103006-ac9b-4901-97a1-47a0e0ee5320 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:36.384206 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1069/2151] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:36 2022] PUT /identity/v3/projects/5cc7e6d82b1341fab2008f5d338efb16/users/4114064accaa45ce9e993cffed949114/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:36.395428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e2c62-9c1b-4012-8748-63fb357236c1 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:36.395722 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e2c62-9c1b-4012-8748-63fb357236c1 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:36.396023 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e2c62-9c1b-4012-8748-63fb357236c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:36.430387 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-963e2c62-9c1b-4012-8748-63fb357236c1 None None] MFA Rules not processed for user `4114064accaa45ce9e993cffed949114`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:36.431541 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-963e2c62-9c1b-4012-8748-63fb357236c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:36.509098 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1083/2152] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:36 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:36.521203 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e103006-ac9b-4901-97a1-47a0e0ee5320 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:36.523191 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-4e103006-ac9b-4901-97a1-47a0e0ee5320 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:36.634067 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e103006-ac9b-4901-97a1-47a0e0ee5320 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:36.642589 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47ba44-cf85-4fe6-b8c9-5ad3825fb0c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:36.643813 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47ba44-cf85-4fe6-b8c9-5ad3825fb0c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:36.644311 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47ba44-cf85-4fe6-b8c9-5ad3825fb0c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:36.644860 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47ba44-cf85-4fe6-b8c9-5ad3825fb0c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:36.660739 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a47ba44-cf85-4fe6-b8c9-5ad3825fb0c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:36.663736 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a47ba44-cf85-4fe6-b8c9-5ad3825fb0c3 service cinder] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:36.683240 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1070/2153] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:36 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:39.188023 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d1791-bcf7-4e39-9bd6-f39665519e7f None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:39.188636 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d1791-bcf7-4e39-9bd6-f39665519e7f None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:39.188636 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d1791-bcf7-4e39-9bd6-f39665519e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:39.342363 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-133d1791-bcf7-4e39-9bd6-f39665519e7f None None] MFA Rules not processed for user `1c85840c4d104b04b6ed577cce2ca2ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:39.343685 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-133d1791-bcf7-4e39-9bd6-f39665519e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:39.364541 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:39.425738 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1084/2154] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:46:41.888723 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47ba44-cf85-4fe6-b8c9-5ad3825fb0c3 service cinder] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:41.994755 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:42.003121 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47ba44-cf85-4fe6-b8c9-5ad3825fb0c3 service cinder] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:42.017599 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa57452f-36a3-4217-b5da-5a3df8f0c4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:42.019144 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa57452f-36a3-4217-b5da-5a3df8f0c4c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:42.019522 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa57452f-36a3-4217-b5da-5a3df8f0c4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:42.019879 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa57452f-36a3-4217-b5da-5a3df8f0c4c7 admin admin] PATH_INFO: `/v3/users/b5c56c037b3740c9b79456d755145841` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:42.022466 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa57452f-36a3-4217-b5da-5a3df8f0c4c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5c56c037b3740c9b79456d755145841)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:42.026019 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa57452f-36a3-4217-b5da-5a3df8f0c4c7 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:42.260908 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-aa57452f-36a3-4217-b5da-5a3df8f0c4c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5c56c037b3740c9b79456d755145841'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:42.269128 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-aa57452f-36a3-4217-b5da-5a3df8f0c4c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5c56c037b3740c9b79456d755145841'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:42.269872 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-aa57452f-36a3-4217-b5da-5a3df8f0c4c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5c56c037b3740c9b79456d755145841'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:42.366555 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-aa57452f-36a3-4217-b5da-5a3df8f0c4c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5c56c037b3740c9b79456d755145841'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:42.389349 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-aa57452f-36a3-4217-b5da-5a3df8f0c4c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5c56c037b3740c9b79456d755145841'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:42.398425 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1071/2155] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:41 2022] DELETE /identity/v3/users/b5c56c037b3740c9b79456d755145841 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:42.403182 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c45fc2-5080-4168-b04a-fbe427b4ea19 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:42.403815 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c45fc2-5080-4168-b04a-fbe427b4ea19 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:42.403815 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c45fc2-5080-4168-b04a-fbe427b4ea19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:42.426941 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-28c45fc2-5080-4168-b04a-fbe427b4ea19 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:42.428168 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-28c45fc2-5080-4168-b04a-fbe427b4ea19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:42.513874 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1085/2156] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:42.523103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa57452f-36a3-4217-b5da-5a3df8f0c4c7 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:42.525390 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-aa57452f-36a3-4217-b5da-5a3df8f0c4c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:42.638910 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa57452f-36a3-4217-b5da-5a3df8f0c4c7 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:42.654104 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e142156f-cb82-421b-baff-0f55c948bfaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:42.654753 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e142156f-cb82-421b-baff-0f55c948bfaf service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:42.655189 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e142156f-cb82-421b-baff-0f55c948bfaf service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:42.655477 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e142156f-cb82-421b-baff-0f55c948bfaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:42.673768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e142156f-cb82-421b-baff-0f55c948bfaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:42.673768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e142156f-cb82-421b-baff-0f55c948bfaf service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:42.685873 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1072/2157] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:43.042076 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c45fc2-5080-4168-b04a-fbe427b4ea19 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:43.044601 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-28c45fc2-5080-4168-b04a-fbe427b4ea19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:43.063390 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c45fc2-5080-4168-b04a-fbe427b4ea19 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:43.069100 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea06a536-1585-405d-9bc1-f70be0b699ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:43.069948 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea06a536-1585-405d-9bc1-f70be0b699ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:43.070244 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea06a536-1585-405d-9bc1-f70be0b699ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:43.071194 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea06a536-1585-405d-9bc1-f70be0b699ea admin admin] PATH_INFO: `/v3/users/2e4ff010ef794b3c972488e781e0b19a` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:43.072868 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea06a536-1585-405d-9bc1-f70be0b699ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e4ff010ef794b3c972488e781e0b19a)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:43.075142 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea06a536-1585-405d-9bc1-f70be0b699ea admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:43.097139 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e142156f-cb82-421b-baff-0f55c948bfaf service neutron] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:43.098794 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-e142156f-cb82-421b-baff-0f55c948bfaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:43.119619 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e142156f-cb82-421b-baff-0f55c948bfaf service neutron] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:43.127261 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf3c400-651c-4c75-8e96-a9d469d0d464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:43.129827 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3c400-651c-4c75-8e96-a9d469d0d464 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:43.130343 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3c400-651c-4c75-8e96-a9d469d0d464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:43.130749 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3c400-651c-4c75-8e96-a9d469d0d464 admin admin] PATH_INFO: `/v3/users/be903033452a42eeb3905b5dc214945d` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:43.132623 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf3c400-651c-4c75-8e96-a9d469d0d464 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be903033452a42eeb3905b5dc214945d)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:43.134637 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf3c400-651c-4c75-8e96-a9d469d0d464 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:43.395938 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-ea06a536-1585-405d-9bc1-f70be0b699ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e4ff010ef794b3c972488e781e0b19a'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:43.421267 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-4bf3c400-651c-4c75-8e96-a9d469d0d464 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be903033452a42eeb3905b5dc214945d'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:43.445391 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-4bf3c400-651c-4c75-8e96-a9d469d0d464 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be903033452a42eeb3905b5dc214945d'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:43.446668 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-4bf3c400-651c-4c75-8e96-a9d469d0d464 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be903033452a42eeb3905b5dc214945d'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:43.505470 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-ea06a536-1585-405d-9bc1-f70be0b699ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e4ff010ef794b3c972488e781e0b19a'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:43.524813 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-ea06a536-1585-405d-9bc1-f70be0b699ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e4ff010ef794b3c972488e781e0b19a'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:43.560112 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-ea06a536-1585-405d-9bc1-f70be0b699ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e4ff010ef794b3c972488e781e0b19a'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:43.561801 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-ea06a536-1585-405d-9bc1-f70be0b699ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e4ff010ef794b3c972488e781e0b19a'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:43.577164 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1086/2158] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:43 2022] DELETE /identity/v3/users/2e4ff010ef794b3c972488e781e0b19a => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:43.579185 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea06a536-1585-405d-9bc1-f70be0b699ea admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:43.582105 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-ea06a536-1585-405d-9bc1-f70be0b699ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:43.622829 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-4bf3c400-651c-4c75-8e96-a9d469d0d464 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be903033452a42eeb3905b5dc214945d'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:43.633616 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:43.647231 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-4bf3c400-651c-4c75-8e96-a9d469d0d464 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be903033452a42eeb3905b5dc214945d'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:43.650247 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:43.668797 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1073/2159] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:43 2022] DELETE /identity/v3/users/be903033452a42eeb3905b5dc214945d => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:43.670651 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f1fdfc-332d-4ead-b53b-57cbba452434 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:43.671150 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f1fdfc-332d-4ead-b53b-57cbba452434 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:43.671914 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f1fdfc-332d-4ead-b53b-57cbba452434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:43.696262 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea06a536-1585-405d-9bc1-f70be0b699ea admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:43.714199 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-d6f1fdfc-332d-4ead-b53b-57cbba452434 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:43.716484 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-d6f1fdfc-332d-4ead-b53b-57cbba452434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:43.836645 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1074/2160] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:43.836645 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf26ad-7622-4e86-b0a3-3da186a91bf5 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:43.836645 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf26ad-7622-4e86-b0a3-3da186a91bf5 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:43.836645 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf26ad-7622-4e86-b0a3-3da186a91bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:43.846073 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335fdef7-2805-4d5d-8cb0-7f4424907d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:43.846554 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335fdef7-2805-4d5d-8cb0-7f4424907d67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:43.847040 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335fdef7-2805-4d5d-8cb0-7f4424907d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:43.847525 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335fdef7-2805-4d5d-8cb0-7f4424907d67 admin admin] PATH_INFO: `/v3/projects/6c07362c048745f5a9c69b821258d558` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:43.850897 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335fdef7-2805-4d5d-8cb0-7f4424907d67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c07362c048745f5a9c69b821258d558)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:43.858524 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-7fbf26ad-7622-4e86-b0a3-3da186a91bf5 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:43.859771 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-7fbf26ad-7622-4e86-b0a3-3da186a91bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:43.865734 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335fdef7-2805-4d5d-8cb0-7f4424907d67 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:43.873317 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1075/2161] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:43.873317 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf26ad-7622-4e86-b0a3-3da186a91bf5 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:43.880962 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-7fbf26ad-7622-4e86-b0a3-3da186a91bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:43.999606 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf26ad-7622-4e86-b0a3-3da186a91bf5 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:44.024890 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-335fdef7-2805-4d5d-8cb0-7f4424907d67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c07362c048745f5a9c69b821258d558', 'request_id': 'req-335fdef7-2805-4d5d-8cb0-7f4424907d67'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:44.053913 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-335fdef7-2805-4d5d-8cb0-7f4424907d67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c07362c048745f5a9c69b821258d558', 'request_id': 'req-335fdef7-2805-4d5d-8cb0-7f4424907d67'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:44.104626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e4364-130d-4ccb-9e3c-63d921dc9ad0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:44.105533 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e4364-130d-4ccb-9e3c-63d921dc9ad0 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:44.105831 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e4364-130d-4ccb-9e3c-63d921dc9ad0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:44.106413 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e4364-130d-4ccb-9e3c-63d921dc9ad0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:44.165582 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-335fdef7-2805-4d5d-8cb0-7f4424907d67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c07362c048745f5a9c69b821258d558', 'request_id': 'req-335fdef7-2805-4d5d-8cb0-7f4424907d67'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:44.195193 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-335fdef7-2805-4d5d-8cb0-7f4424907d67 admin admin] The token cache is being invalidate because project 6c07362c048745f5a9c69b821258d558 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:44.196776 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-335fdef7-2805-4d5d-8cb0-7f4424907d67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:44.199533 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-335fdef7-2805-4d5d-8cb0-7f4424907d67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:44.200790 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425e4364-130d-4ccb-9e3c-63d921dc9ad0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:44.202400 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425e4364-130d-4ccb-9e3c-63d921dc9ad0 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:44.203914 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1087/2162] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:43 2022] DELETE /identity/v3/projects/6c07362c048745f5a9c69b821258d558 => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:44.206707 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335fdef7-2805-4d5d-8cb0-7f4424907d67 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:44.215335 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1076/2163] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:44.217266 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e4364-130d-4ccb-9e3c-63d921dc9ad0 service neutron] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:44.222028 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-335fdef7-2805-4d5d-8cb0-7f4424907d67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:44.231070 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e4364-130d-4ccb-9e3c-63d921dc9ad0 service neutron] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:44.243128 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335fdef7-2805-4d5d-8cb0-7f4424907d67 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:44.245442 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b8621-4049-49fe-b032-c6dd1c75a3b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:44.246088 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b8621-4049-49fe-b032-c6dd1c75a3b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:44.246371 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b8621-4049-49fe-b032-c6dd1c75a3b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:44.246643 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b8621-4049-49fe-b032-c6dd1c75a3b4 admin admin] PATH_INFO: `/v3/users/4114064accaa45ce9e993cffed949114` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:44.248261 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6b8621-4049-49fe-b032-c6dd1c75a3b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4114064accaa45ce9e993cffed949114)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:44.250282 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6b8621-4049-49fe-b032-c6dd1c75a3b4 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:44.255921 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ca425-051c-4113-9403-efd7dbb615bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:44.256599 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ca425-051c-4113-9403-efd7dbb615bc service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:44.256881 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ca425-051c-4113-9403-efd7dbb615bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:44.266075 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ca425-051c-4113-9403-efd7dbb615bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:44.266075 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-6b3ca425-051c-4113-9403-efd7dbb615bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:44.290429 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3ca425-051c-4113-9403-efd7dbb615bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:44.296448 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3ca425-051c-4113-9403-efd7dbb615bc service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:44.317096 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1088/2164] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:44.491827 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-5c6b8621-4049-49fe-b032-c6dd1c75a3b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4114064accaa45ce9e993cffed949114'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:44.503807 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-5c6b8621-4049-49fe-b032-c6dd1c75a3b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4114064accaa45ce9e993cffed949114'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:44.504843 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-5c6b8621-4049-49fe-b032-c6dd1c75a3b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4114064accaa45ce9e993cffed949114'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:44.609219 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-5c6b8621-4049-49fe-b032-c6dd1c75a3b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4114064accaa45ce9e993cffed949114'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:44.631594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-5c6b8621-4049-49fe-b032-c6dd1c75a3b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4114064accaa45ce9e993cffed949114'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:44.641821 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1077/2165] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:44 2022] DELETE /identity/v3/users/4114064accaa45ce9e993cffed949114 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:45.072542 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ca425-051c-4113-9403-efd7dbb615bc service neutron] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:45.075159 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-6b3ca425-051c-4113-9403-efd7dbb615bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:45.103116 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b8621-4049-49fe-b032-c6dd1c75a3b4 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:45.105188 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-5c6b8621-4049-49fe-b032-c6dd1c75a3b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:45.117114 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ca425-051c-4113-9403-efd7dbb615bc service neutron] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:45.229774 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a13717-2b6c-4844-a398-74d8c55b6dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:45.230797 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a13717-2b6c-4844-a398-74d8c55b6dcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:45.231258 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a13717-2b6c-4844-a398-74d8c55b6dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:45.231724 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a13717-2b6c-4844-a398-74d8c55b6dcd admin admin] PATH_INFO: `/v3/projects/39fcd5de93c744388a2f41a0daf959cd` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:45.233365 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a13717-2b6c-4844-a398-74d8c55b6dcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=39fcd5de93c744388a2f41a0daf959cd)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:45.235624 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a13717-2b6c-4844-a398-74d8c55b6dcd admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:45.241668 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b8621-4049-49fe-b032-c6dd1c75a3b4 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:45.257049 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af208ed-6ee4-41bc-85ac-df2053e512be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:45.258203 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af208ed-6ee4-41bc-85ac-df2053e512be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:45.259074 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af208ed-6ee4-41bc-85ac-df2053e512be admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:45.259444 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af208ed-6ee4-41bc-85ac-df2053e512be admin admin] PATH_INFO: `/v3/projects/6738e05eed4849e5b67d8b8d30f22269` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:45.260932 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af208ed-6ee4-41bc-85ac-df2053e512be admin admin] RBAC: Authorizing `identity:delete_project(project_id=6738e05eed4849e5b67d8b8d30f22269)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:45.263184 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af208ed-6ee4-41bc-85ac-df2053e512be admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:45.452238 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-5af208ed-6ee4-41bc-85ac-df2053e512be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6738e05eed4849e5b67d8b8d30f22269', 'request_id': 'req-5af208ed-6ee4-41bc-85ac-df2053e512be'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:45.458230 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-50a13717-2b6c-4844-a398-74d8c55b6dcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39fcd5de93c744388a2f41a0daf959cd', 'request_id': 'req-50a13717-2b6c-4844-a398-74d8c55b6dcd'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:45.459831 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-5af208ed-6ee4-41bc-85ac-df2053e512be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6738e05eed4849e5b67d8b8d30f22269', 'request_id': 'req-5af208ed-6ee4-41bc-85ac-df2053e512be'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:45.467200 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-50a13717-2b6c-4844-a398-74d8c55b6dcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39fcd5de93c744388a2f41a0daf959cd', 'request_id': 'req-50a13717-2b6c-4844-a398-74d8c55b6dcd'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:45.577048 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-5af208ed-6ee4-41bc-85ac-df2053e512be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6738e05eed4849e5b67d8b8d30f22269', 'request_id': 'req-5af208ed-6ee4-41bc-85ac-df2053e512be'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:45.592204 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-5af208ed-6ee4-41bc-85ac-df2053e512be admin admin] The token cache is being invalidate because project 6738e05eed4849e5b67d8b8d30f22269 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:45.592344 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-5af208ed-6ee4-41bc-85ac-df2053e512be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:45.593025 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-5af208ed-6ee4-41bc-85ac-df2053e512be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:45.594530 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1078/2166] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:45 2022] DELETE /identity/v3/projects/6738e05eed4849e5b67d8b8d30f22269 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:46:45.595288 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af208ed-6ee4-41bc-85ac-df2053e512be admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:45.596669 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-5af208ed-6ee4-41bc-85ac-df2053e512be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:45.679689 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af208ed-6ee4-41bc-85ac-df2053e512be admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:45.700490 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:45.707786 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-50a13717-2b6c-4844-a398-74d8c55b6dcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39fcd5de93c744388a2f41a0daf959cd', 'request_id': 'req-50a13717-2b6c-4844-a398-74d8c55b6dcd'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:45.722295 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-50a13717-2b6c-4844-a398-74d8c55b6dcd admin admin] The token cache is being invalidate because project 39fcd5de93c744388a2f41a0daf959cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:45.722976 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-50a13717-2b6c-4844-a398-74d8c55b6dcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:45.723887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-50a13717-2b6c-4844-a398-74d8c55b6dcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:45.728887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1089/2167] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:45 2022] DELETE /identity/v3/projects/39fcd5de93c744388a2f41a0daf959cd => generated 0 bytes in 654 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:45.730511 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a13717-2b6c-4844-a398-74d8c55b6dcd admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:45.733082 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-50a13717-2b6c-4844-a398-74d8c55b6dcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:45.765883 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6849403a-3e62-405b-998b-bc764a7cd26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:45.767048 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849403a-3e62-405b-998b-bc764a7cd26f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:45.768368 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849403a-3e62-405b-998b-bc764a7cd26f admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:45.768853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849403a-3e62-405b-998b-bc764a7cd26f admin admin] PATH_INFO: `/v3/projects/5cc7e6d82b1341fab2008f5d338efb16` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:45.771037 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6849403a-3e62-405b-998b-bc764a7cd26f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cc7e6d82b1341fab2008f5d338efb16)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:45.775703 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6849403a-3e62-405b-998b-bc764a7cd26f admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:45.787824 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:45.850376 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a13717-2b6c-4844-a398-74d8c55b6dcd admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:45.857011 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c8e818-4448-4ed9-8d2b-e00af537a31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:45.858005 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8e818-4448-4ed9-8d2b-e00af537a31d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:45.858437 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8e818-4448-4ed9-8d2b-e00af537a31d admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:45.858902 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8e818-4448-4ed9-8d2b-e00af537a31d admin admin] PATH_INFO: `/v3/users/16b0404333fa45b0b4645169ab39ff66` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:45.862111 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c8e818-4448-4ed9-8d2b-e00af537a31d admin admin] RBAC: Authorizing `identity:delete_user(user_id=16b0404333fa45b0b4645169ab39ff66)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:45.865664 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c8e818-4448-4ed9-8d2b-e00af537a31d admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:45.919014 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-6849403a-3e62-405b-998b-bc764a7cd26f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cc7e6d82b1341fab2008f5d338efb16', 'request_id': 'req-6849403a-3e62-405b-998b-bc764a7cd26f'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:45.930033 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-6849403a-3e62-405b-998b-bc764a7cd26f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cc7e6d82b1341fab2008f5d338efb16', 'request_id': 'req-6849403a-3e62-405b-998b-bc764a7cd26f'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:45.994343 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-6849403a-3e62-405b-998b-bc764a7cd26f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cc7e6d82b1341fab2008f5d338efb16', 'request_id': 'req-6849403a-3e62-405b-998b-bc764a7cd26f'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:46.008515 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-6849403a-3e62-405b-998b-bc764a7cd26f admin admin] The token cache is being invalidate because project 5cc7e6d82b1341fab2008f5d338efb16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:46.008810 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-6849403a-3e62-405b-998b-bc764a7cd26f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:46.009501 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-6849403a-3e62-405b-998b-bc764a7cd26f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:46.011029 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1079/2168] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:45 2022] DELETE /identity/v3/projects/5cc7e6d82b1341fab2008f5d338efb16 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:46.015773 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201af795-5926-4e4f-95cf-b0d3c3fdd459 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:46.016068 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201af795-5926-4e4f-95cf-b0d3c3fdd459 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:46.016352 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201af795-5926-4e4f-95cf-b0d3c3fdd459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:46.037407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-201af795-5926-4e4f-95cf-b0d3c3fdd459 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:46.038721 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-201af795-5926-4e4f-95cf-b0d3c3fdd459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:46.108613 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1080/2169] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:46:46.120960 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf65342-d90d-42e6-bb8a-dec03b29e619 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:46.121456 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf65342-d90d-42e6-bb8a-dec03b29e619 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:46.121952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf65342-d90d-42e6-bb8a-dec03b29e619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:46.124722 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-a8c8e818-4448-4ed9-8d2b-e00af537a31d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16b0404333fa45b0b4645169ab39ff66'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:46.146923 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-adf65342-d90d-42e6-bb8a-dec03b29e619 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:46.148176 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-adf65342-d90d-42e6-bb8a-dec03b29e619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:46.160563 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1081/2170] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:46:46.163169 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf65342-d90d-42e6-bb8a-dec03b29e619 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:46.177752 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf65342-d90d-42e6-bb8a-dec03b29e619 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:46.183400 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ac4a5-2710-4a0f-bef2-6103812b81f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:46.187953 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ac4a5-2710-4a0f-bef2-6103812b81f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:46.187953 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ac4a5-2710-4a0f-bef2-6103812b81f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:46.187953 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ac4a5-2710-4a0f-bef2-6103812b81f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:46.187953 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239ac4a5-2710-4a0f-bef2-6103812b81f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:46:46.187953 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239ac4a5-2710-4a0f-bef2-6103812b81f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:46.187953 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239ac4a5-2710-4a0f-bef2-6103812b81f5 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:46.201280 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1082/2171] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:46:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:46:46.207034 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ac4a5-2710-4a0f-bef2-6103812b81f5 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:46.208895 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-a8c8e818-4448-4ed9-8d2b-e00af537a31d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16b0404333fa45b0b4645169ab39ff66'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:46.216693 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-a8c8e818-4448-4ed9-8d2b-e00af537a31d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16b0404333fa45b0b4645169ab39ff66'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:46.217498 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ac4a5-2710-4a0f-bef2-6103812b81f5 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:46.222583 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c8dcc-336d-4be7-a903-f10ea0d2ad2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:46.223200 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c8dcc-336d-4be7-a903-f10ea0d2ad2f admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:46.223485 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c8dcc-336d-4be7-a903-f10ea0d2ad2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:46.223764 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c8dcc-336d-4be7-a903-f10ea0d2ad2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:46.224389 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2c8dcc-336d-4be7-a903-f10ea0d2ad2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:46.226336 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2c8dcc-336d-4be7-a903-f10ea0d2ad2f admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:46.231172 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-a8c8e818-4448-4ed9-8d2b-e00af537a31d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16b0404333fa45b0b4645169ab39ff66'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:46.232065 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-a8c8e818-4448-4ed9-8d2b-e00af537a31d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16b0404333fa45b0b4645169ab39ff66'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:46.240250 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1090/2172] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:45 2022] DELETE /identity/v3/users/16b0404333fa45b0b4645169ab39ff66 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:46.338681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1083/2173] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:46 2022] POST /identity/v3/projects => generated 387 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:46:46.339825 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c8e818-4448-4ed9-8d2b-e00af537a31d admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:46.341383 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-a8c8e818-4448-4ed9-8d2b-e00af537a31d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:46.428157 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c8e818-4448-4ed9-8d2b-e00af537a31d admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:46.512761 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19676dda-9ffa-488f-b900-0b9a2093b2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:46.513638 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19676dda-9ffa-488f-b900-0b9a2093b2cd admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:46.514475 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19676dda-9ffa-488f-b900-0b9a2093b2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:46.514475 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19676dda-9ffa-488f-b900-0b9a2093b2cd admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:46.515102 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19676dda-9ffa-488f-b900-0b9a2093b2cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:46.517824 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19676dda-9ffa-488f-b900-0b9a2093b2cd admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:46.713151 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1091/2174] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:46 2022] POST /identity/v3/users => generated 294 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:46.717442 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c8dcc-336d-4be7-a903-f10ea0d2ad2f admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:46.742141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c8dcc-336d-4be7-a903-f10ea0d2ad2f admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:46.749489 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eed2da-6d49-44c4-bd03-0981c36dd031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:46.750326 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eed2da-6d49-44c4-bd03-0981c36dd031 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:46.750619 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eed2da-6d49-44c4-bd03-0981c36dd031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:46.750902 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eed2da-6d49-44c4-bd03-0981c36dd031 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:46.752639 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eed2da-6d49-44c4-bd03-0981c36dd031 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:46.757201 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eed2da-6d49-44c4-bd03-0981c36dd031 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:46.775729 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1084/2175] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:46 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:46.787680 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19676dda-9ffa-488f-b900-0b9a2093b2cd admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:46.797399 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19676dda-9ffa-488f-b900-0b9a2093b2cd admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:46.807203 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c8d040-10f7-4308-8888-efe237c35793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:46.808217 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c8d040-10f7-4308-8888-efe237c35793 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:46.808635 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c8d040-10f7-4308-8888-efe237c35793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:46.809011 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c8d040-10f7-4308-8888-efe237c35793 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:46.810377 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c8d040-10f7-4308-8888-efe237c35793 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:46.814369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c8d040-10f7-4308-8888-efe237c35793 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:46.828304 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:46.831660 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1092/2176] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:46 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:46.835777 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eed2da-6d49-44c4-bd03-0981c36dd031 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:46.851584 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eed2da-6d49-44c4-bd03-0981c36dd031 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:46.857605 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cb09ad-3377-430a-ad1b-bc01ed72bbd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:46.858367 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb09ad-3377-430a-ad1b-bc01ed72bbd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:46.858566 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb09ad-3377-430a-ad1b-bc01ed72bbd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:46.858847 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb09ad-3377-430a-ad1b-bc01ed72bbd4 admin admin] PATH_INFO: `/v3/projects/ab6e0f4a9c7b466abad562f1c13262c0/users/c3621c64c9bc4a22b91f2043633beda7/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:46.879469 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cb09ad-3377-430a-ad1b-bc01ed72bbd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab6e0f4a9c7b466abad562f1c13262c0, user_id=c3621c64c9bc4a22b91f2043633beda7, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:46.882465 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cb09ad-3377-430a-ad1b-bc01ed72bbd4 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:46.991693 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1085/2177] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:46 2022] PUT /identity/v3/projects/ab6e0f4a9c7b466abad562f1c13262c0/users/c3621c64c9bc4a22b91f2043633beda7/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:47.012428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae0616-5899-4af9-8bd1-c560560e95ec None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:47.012428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae0616-5899-4af9-8bd1-c560560e95ec None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:47.012428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae0616-5899-4af9-8bd1-c560560e95ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:47.047301 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-20ae0616-5899-4af9-8bd1-c560560e95ec None None] MFA Rules not processed for user `c3621c64c9bc4a22b91f2043633beda7`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:47.048702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-20ae0616-5899-4af9-8bd1-c560560e95ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:47.089056 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cb09ad-3377-430a-ad1b-bc01ed72bbd4 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:47.092563 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-e5cb09ad-3377-430a-ad1b-bc01ed72bbd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:47.140078 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1093/2178] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:47.145332 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ae0616-5899-4af9-8bd1-c560560e95ec None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:47.174364 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:47.224091 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cb09ad-3377-430a-ad1b-bc01ed72bbd4 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:47.232719 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a6b6db-4d01-46e6-b614-27b80d1028cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:47.233842 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a6b6db-4d01-46e6-b614-27b80d1028cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:47.234346 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a6b6db-4d01-46e6-b614-27b80d1028cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:47.234847 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a6b6db-4d01-46e6-b614-27b80d1028cf admin admin] PATH_INFO: `/v3/projects/2be1f542a9df4f37be9bf2c065c6dda3` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:47.238149 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a6b6db-4d01-46e6-b614-27b80d1028cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=2be1f542a9df4f37be9bf2c065c6dda3)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:47.241860 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ae0616-5899-4af9-8bd1-c560560e95ec None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:47.242299 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a6b6db-4d01-46e6-b614-27b80d1028cf admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:47.250642 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998fdf7d-d063-4973-8f40-c7a9372d775c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:47.252112 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998fdf7d-d063-4973-8f40-c7a9372d775c admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:47.252584 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998fdf7d-d063-4973-8f40-c7a9372d775c admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:47.252877 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998fdf7d-d063-4973-8f40-c7a9372d775c admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:47.253506 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998fdf7d-d063-4973-8f40-c7a9372d775c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:47.255496 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998fdf7d-d063-4973-8f40-c7a9372d775c admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:47.397341 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1094/2179] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:47 2022] POST /identity/v3/projects => generated 389 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:47.402278 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998fdf7d-d063-4973-8f40-c7a9372d775c admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:47.417738 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-94a6b6db-4d01-46e6-b614-27b80d1028cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2be1f542a9df4f37be9bf2c065c6dda3', 'request_id': 'req-94a6b6db-4d01-46e6-b614-27b80d1028cf'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:47.426090 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-94a6b6db-4d01-46e6-b614-27b80d1028cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2be1f542a9df4f37be9bf2c065c6dda3', 'request_id': 'req-94a6b6db-4d01-46e6-b614-27b80d1028cf'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:47.484567 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998fdf7d-d063-4973-8f40-c7a9372d775c admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:47.490317 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4bd62c-f762-48dc-baed-e82e596c6209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:47.490931 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4bd62c-f762-48dc-baed-e82e596c6209 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:47.491226 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4bd62c-f762-48dc-baed-e82e596c6209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:47.491496 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4bd62c-f762-48dc-baed-e82e596c6209 admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:47.492127 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4bd62c-f762-48dc-baed-e82e596c6209 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:47.494292 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4bd62c-f762-48dc-baed-e82e596c6209 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:47.520151 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-94a6b6db-4d01-46e6-b614-27b80d1028cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2be1f542a9df4f37be9bf2c065c6dda3', 'request_id': 'req-94a6b6db-4d01-46e6-b614-27b80d1028cf'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:47.531771 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-94a6b6db-4d01-46e6-b614-27b80d1028cf admin admin] The token cache is being invalidate because project 2be1f542a9df4f37be9bf2c065c6dda3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:47.531771 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-94a6b6db-4d01-46e6-b614-27b80d1028cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:47.532288 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-94a6b6db-4d01-46e6-b614-27b80d1028cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:47.535234 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1086/2180] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:47 2022] DELETE /identity/v3/projects/2be1f542a9df4f37be9bf2c065c6dda3 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:47.542505 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11771ff4-6d12-4909-8284-4097bcfd000d None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:47.542846 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11771ff4-6d12-4909-8284-4097bcfd000d None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:47.543051 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11771ff4-6d12-4909-8284-4097bcfd000d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:47.563975 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-11771ff4-6d12-4909-8284-4097bcfd000d None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:47.565317 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-11771ff4-6d12-4909-8284-4097bcfd000d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:47.574557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1087/2181] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:46:47.583388 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ab851-3ae8-4f6b-9af3-2b5be973a569 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:47.583673 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ab851-3ae8-4f6b-9af3-2b5be973a569 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:47.583954 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ab851-3ae8-4f6b-9af3-2b5be973a569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:47.596503 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:47.606809 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-450ab851-3ae8-4f6b-9af3-2b5be973a569 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:47.608002 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-450ab851-3ae8-4f6b-9af3-2b5be973a569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:47.616303 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1088/2182] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:46:47.620588 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ab851-3ae8-4f6b-9af3-2b5be973a569 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:47.631062 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ab851-3ae8-4f6b-9af3-2b5be973a569 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:47.636809 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914bf73f-77c9-4b92-99da-855620b50a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:47.638002 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914bf73f-77c9-4b92-99da-855620b50a15 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:47.638796 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914bf73f-77c9-4b92-99da-855620b50a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:47.638796 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914bf73f-77c9-4b92-99da-855620b50a15 admin admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:47.639494 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914bf73f-77c9-4b92-99da-855620b50a15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:46:47.639787 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914bf73f-77c9-4b92-99da-855620b50a15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:47.641527 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914bf73f-77c9-4b92-99da-855620b50a15 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:47.644233 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1095/2183] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:47 2022] POST /identity/v3/users => generated 301 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:46:47.645159 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4bd62c-f762-48dc-baed-e82e596c6209 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:47.646907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-0e4bd62c-f762-48dc-baed-e82e596c6209 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:47.661466 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1089/2184] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:46:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:46:47.662764 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914bf73f-77c9-4b92-99da-855620b50a15 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:47.675827 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4bd62c-f762-48dc-baed-e82e596c6209 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:47.677324 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914bf73f-77c9-4b92-99da-855620b50a15 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:47.684113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0093bf4-b6ff-4746-9865-b3e2a021de2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:47.684793 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0093bf4-b6ff-4746-9865-b3e2a021de2e admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:47.685080 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0093bf4-b6ff-4746-9865-b3e2a021de2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:47.685362 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0093bf4-b6ff-4746-9865-b3e2a021de2e admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:47.686005 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0093bf4-b6ff-4746-9865-b3e2a021de2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:47.688368 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0093bf4-b6ff-4746-9865-b3e2a021de2e admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:47.689981 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e69fbbb-ca3d-418b-8ab5-950bb243505c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:47.693468 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e69fbbb-ca3d-418b-8ab5-950bb243505c admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:47.693937 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e69fbbb-ca3d-418b-8ab5-950bb243505c admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:47.694319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e69fbbb-ca3d-418b-8ab5-950bb243505c admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:47.695717 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e69fbbb-ca3d-418b-8ab5-950bb243505c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:47.697859 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e69fbbb-ca3d-418b-8ab5-950bb243505c admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:47.719685 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1096/2185] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:47 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:47.722325 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e69fbbb-ca3d-418b-8ab5-950bb243505c admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:47.740321 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e69fbbb-ca3d-418b-8ab5-950bb243505c admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:47.746599 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d047db-275e-4d63-9f7c-d4a7bdfd43c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:47.747294 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d047db-275e-4d63-9f7c-d4a7bdfd43c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:47.747576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d047db-275e-4d63-9f7c-d4a7bdfd43c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:47.747858 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d047db-275e-4d63-9f7c-d4a7bdfd43c9 admin admin] PATH_INFO: `/v3/projects/e869874ba4e74ef39b3143b5bb187d00/users/880af75de0cf4818bcf9281ff3059f85/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:47.768061 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d047db-275e-4d63-9f7c-d4a7bdfd43c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e869874ba4e74ef39b3143b5bb187d00, user_id=880af75de0cf4818bcf9281ff3059f85, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:47.772133 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d047db-275e-4d63-9f7c-d4a7bdfd43c9 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:47.839390 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1090/2186] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:47 2022] POST /identity/v3/projects => generated 381 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:47.843925 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0093bf4-b6ff-4746-9865-b3e2a021de2e admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:47.889874 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1097/2187] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:47 2022] PUT /identity/v3/projects/e869874ba4e74ef39b3143b5bb187d00/users/880af75de0cf4818bcf9281ff3059f85/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:47.895540 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d047db-275e-4d63-9f7c-d4a7bdfd43c9 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:47.932716 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:47.950314 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0093bf4-b6ff-4746-9865-b3e2a021de2e admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:48.013525 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d047db-275e-4d63-9f7c-d4a7bdfd43c9 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:48.020369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12cec4-b329-4714-9cbb-9237ab5fa45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:48.021163 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12cec4-b329-4714-9cbb-9237ab5fa45d admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:48.021459 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12cec4-b329-4714-9cbb-9237ab5fa45d admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:48.021732 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12cec4-b329-4714-9cbb-9237ab5fa45d admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:48.022391 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff12cec4-b329-4714-9cbb-9237ab5fa45d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:48.024030 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff12cec4-b329-4714-9cbb-9237ab5fa45d admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:48.026587 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b8eff-9a9c-4ffb-a054-14fe46f98d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:48.027273 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b8eff-9a9c-4ffb-a054-14fe46f98d46 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:48.032174 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b8eff-9a9c-4ffb-a054-14fe46f98d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:48.032174 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b8eff-9a9c-4ffb-a054-14fe46f98d46 admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:48.032174 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3b8eff-9a9c-4ffb-a054-14fe46f98d46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:48.032174 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3b8eff-9a9c-4ffb-a054-14fe46f98d46 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:48.042918 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1098/2188] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:47 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:48.047301 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12cec4-b329-4714-9cbb-9237ab5fa45d admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:48.062527 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12cec4-b329-4714-9cbb-9237ab5fa45d admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:48.068237 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a4d3e1-f2b3-4490-8721-a005f7a321ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:48.068955 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4d3e1-f2b3-4490-8721-a005f7a321ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:48.069237 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4d3e1-f2b3-4490-8721-a005f7a321ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:48.069570 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4d3e1-f2b3-4490-8721-a005f7a321ab admin admin] PATH_INFO: `/v3/projects/e869874ba4e74ef39b3143b5bb187d00/users/880af75de0cf4818bcf9281ff3059f85/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:48.071547 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a4d3e1-f2b3-4490-8721-a005f7a321ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=e869874ba4e74ef39b3143b5bb187d00, user_id=880af75de0cf4818bcf9281ff3059f85, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:48.075561 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a4d3e1-f2b3-4490-8721-a005f7a321ab admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:48.090810 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1099/2189] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:48 2022] PUT /identity/v3/projects/e869874ba4e74ef39b3143b5bb187d00/users/880af75de0cf4818bcf9281ff3059f85/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:48.101396 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9df361-0ae4-41bd-8927-4ab3b6335282 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:48.102220 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9df361-0ae4-41bd-8927-4ab3b6335282 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:48.102511 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9df361-0ae4-41bd-8927-4ab3b6335282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:48.137582 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-7c9df361-0ae4-41bd-8927-4ab3b6335282 None None] MFA Rules not processed for user `880af75de0cf4818bcf9281ff3059f85`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:48.138646 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-7c9df361-0ae4-41bd-8927-4ab3b6335282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:48.199257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1091/2190] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:47 2022] POST /identity/v3/users => generated 291 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:46:48.203790 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b8eff-9a9c-4ffb-a054-14fe46f98d46 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:48.224644 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b8eff-9a9c-4ffb-a054-14fe46f98d46 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:48.236806 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1100/2191] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:48 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:46:48.247308 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9df361-0ae4-41bd-8927-4ab3b6335282 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:48.250354 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-7c9df361-0ae4-41bd-8927-4ab3b6335282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:48.313597 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31beca62-b6fd-4f61-8845-cf776269f765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:48.315039 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31beca62-b6fd-4f61-8845-cf776269f765 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:48.315607 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31beca62-b6fd-4f61-8845-cf776269f765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:48.315972 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31beca62-b6fd-4f61-8845-cf776269f765 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:48.316705 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31beca62-b6fd-4f61-8845-cf776269f765 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:48.318369 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31beca62-b6fd-4f61-8845-cf776269f765 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:48.331036 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1092/2192] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:48 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:48.335149 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31beca62-b6fd-4f61-8845-cf776269f765 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:48.345853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31beca62-b6fd-4f61-8845-cf776269f765 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:48.350762 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d879025-06d5-4f6f-a69c-c74de51da7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:48.351373 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d879025-06d5-4f6f-a69c-c74de51da7e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:48.351654 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d879025-06d5-4f6f-a69c-c74de51da7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:48.351932 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d879025-06d5-4f6f-a69c-c74de51da7e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:48.352821 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d879025-06d5-4f6f-a69c-c74de51da7e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:48.354423 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d879025-06d5-4f6f-a69c-c74de51da7e3 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:48.362204 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9df361-0ae4-41bd-8927-4ab3b6335282 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:48.365229 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1093/2193] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:46:48.365950 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df590b4f-403f-4f26-adcb-ec2be696924a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:48.366559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df590b4f-403f-4f26-adcb-ec2be696924a service cinder] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:48.366837 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df590b4f-403f-4f26-adcb-ec2be696924a service cinder] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:48.367105 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df590b4f-403f-4f26-adcb-ec2be696924a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:48.369304 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d879025-06d5-4f6f-a69c-c74de51da7e3 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:48.379139 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df590b4f-403f-4f26-adcb-ec2be696924a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:48.379640 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d879025-06d5-4f6f-a69c-c74de51da7e3 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:48.380807 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df590b4f-403f-4f26-adcb-ec2be696924a service cinder] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:48.385027 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf76c6f-b940-4d6b-a9ee-94eab93a0d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:48.385726 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf76c6f-b940-4d6b-a9ee-94eab93a0d3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:48.386006 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf76c6f-b940-4d6b-a9ee-94eab93a0d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:48.386346 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf76c6f-b940-4d6b-a9ee-94eab93a0d3e admin admin] PATH_INFO: `/v3/projects/fcd2a21b6e6b4bbba070872967d3ded2/users/7dfa839e456a4fc58be3ae060de0186d/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:48.391477 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1101/2194] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:48 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:46:48.403658 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf76c6f-b940-4d6b-a9ee-94eab93a0d3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcd2a21b6e6b4bbba070872967d3ded2, user_id=7dfa839e456a4fc58be3ae060de0186d, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:48.406357 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf76c6f-b940-4d6b-a9ee-94eab93a0d3e admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:48.426128 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:48.497122 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1094/2195] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:48 2022] PUT /identity/v3/projects/fcd2a21b6e6b4bbba070872967d3ded2/users/7dfa839e456a4fc58be3ae060de0186d/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:48.504644 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:48.514522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e6882-502f-42f2-b59e-f47a206912b7 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:48.514870 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e6882-502f-42f2-b59e-f47a206912b7 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:48.515219 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e6882-502f-42f2-b59e-f47a206912b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:48.543289 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf76c6f-b940-4d6b-a9ee-94eab93a0d3e admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:48.552592 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-be3e6882-502f-42f2-b59e-f47a206912b7 None None] MFA Rules not processed for user `7dfa839e456a4fc58be3ae060de0186d`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:48.556108 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-be3e6882-502f-42f2-b59e-f47a206912b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:48.618758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf76c6f-b940-4d6b-a9ee-94eab93a0d3e admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:48.623806 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7304512-0856-4185-bca7-f70309bee1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:48.624474 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7304512-0856-4185-bca7-f70309bee1b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:48.624762 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7304512-0856-4185-bca7-f70309bee1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:48.625046 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7304512-0856-4185-bca7-f70309bee1b3 admin admin] PATH_INFO: `/v3/users/c3621c64c9bc4a22b91f2043633beda7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:48.627519 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7304512-0856-4185-bca7-f70309bee1b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3621c64c9bc4a22b91f2043633beda7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:48.629521 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7304512-0856-4185-bca7-f70309bee1b3 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:48.639091 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1102/2196] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:48.650933 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e6882-502f-42f2-b59e-f47a206912b7 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:48.706358 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e6882-502f-42f2-b59e-f47a206912b7 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:48.709920 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeff2781-d515-4661-9459-3b087ca65792 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:48.710525 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeff2781-d515-4661-9459-3b087ca65792 service cinder] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:48.710803 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeff2781-d515-4661-9459-3b087ca65792 service cinder] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:48.711078 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeff2781-d515-4661-9459-3b087ca65792 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:48.719695 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeff2781-d515-4661-9459-3b087ca65792 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:48.721774 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeff2781-d515-4661-9459-3b087ca65792 service cinder] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:48.730759 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1103/2197] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:48 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:48.752086 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeff2781-d515-4661-9459-3b087ca65792 service cinder] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:48.763058 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeff2781-d515-4661-9459-3b087ca65792 service cinder] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:48.767674 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ab5fe1-3357-4029-bee2-852f53148bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:48.768288 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab5fe1-3357-4029-bee2-852f53148bb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:48.768594 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab5fe1-3357-4029-bee2-852f53148bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:48.768867 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab5fe1-3357-4029-bee2-852f53148bb5 admin admin] PATH_INFO: `/v3/users/7dfa839e456a4fc58be3ae060de0186d` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:48.770004 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ab5fe1-3357-4029-bee2-852f53148bb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dfa839e456a4fc58be3ae060de0186d)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:48.771905 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ab5fe1-3357-4029-bee2-852f53148bb5 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:48.822900 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-f7304512-0856-4185-bca7-f70309bee1b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3621c64c9bc4a22b91f2043633beda7'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:48.830535 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-f7304512-0856-4185-bca7-f70309bee1b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3621c64c9bc4a22b91f2043633beda7'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:48.831458 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-f7304512-0856-4185-bca7-f70309bee1b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3621c64c9bc4a22b91f2043633beda7'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:48.942862 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-f7304512-0856-4185-bca7-f70309bee1b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3621c64c9bc4a22b91f2043633beda7'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:48.957367 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-f7304512-0856-4185-bca7-f70309bee1b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3621c64c9bc4a22b91f2043633beda7'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:48.962738 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1095/2198] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:48 2022] DELETE /identity/v3/users/c3621c64c9bc4a22b91f2043633beda7 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:46:48.967252 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa889083-7e39-4389-bbde-d5e1f9e15164 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:48.967539 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa889083-7e39-4389-bbde-d5e1f9e15164 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:48.967816 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa889083-7e39-4389-bbde-d5e1f9e15164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:48.991103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-aa889083-7e39-4389-bbde-d5e1f9e15164 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:48.992345 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-aa889083-7e39-4389-bbde-d5e1f9e15164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:48.997207 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-56ab5fe1-3357-4029-bee2-852f53148bb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dfa839e456a4fc58be3ae060de0186d'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:49.015505 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:49.062064 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1096/2199] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:46:49.070031 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-56ab5fe1-3357-4029-bee2-852f53148bb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dfa839e456a4fc58be3ae060de0186d'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:49.072937 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa889083-7e39-4389-bbde-d5e1f9e15164 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:49.074787 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-aa889083-7e39-4389-bbde-d5e1f9e15164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:49.080509 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-56ab5fe1-3357-4029-bee2-852f53148bb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dfa839e456a4fc58be3ae060de0186d'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:49.096687 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-56ab5fe1-3357-4029-bee2-852f53148bb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dfa839e456a4fc58be3ae060de0186d'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:49.097436 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-56ab5fe1-3357-4029-bee2-852f53148bb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dfa839e456a4fc58be3ae060de0186d'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:49.113767 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1104/2200] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:48 2022] DELETE /identity/v3/users/7dfa839e456a4fc58be3ae060de0186d => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:49.118502 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d81026-3c5a-4a83-b935-e7fbc69b954d None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:49.118792 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d81026-3c5a-4a83-b935-e7fbc69b954d None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:49.119065 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d81026-3c5a-4a83-b935-e7fbc69b954d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:49.141402 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa889083-7e39-4389-bbde-d5e1f9e15164 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:49.142786 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-26d81026-3c5a-4a83-b935-e7fbc69b954d None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:49.144033 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-26d81026-3c5a-4a83-b935-e7fbc69b954d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:49.154856 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:INSERT=5,keystone:DELETE=3,keystone:SELECT=92 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:49.210507 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a2d11-194b-4303-ac7f-f83f36d73f48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:49.211359 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a2d11-194b-4303-ac7f-f83f36d73f48 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:49.211936 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a2d11-194b-4303-ac7f-f83f36d73f48 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:49.212546 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a2d11-194b-4303-ac7f-f83f36d73f48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:49.215153 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-2a1a2d11-194b-4303-ac7f-f83f36d73f48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:49.222736 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1105/2201] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:46:49.231664 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1a2d11-194b-4303-ac7f-f83f36d73f48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:49.233475 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1a2d11-194b-4303-ac7f-f83f36d73f48 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:49.233667 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d81026-3c5a-4a83-b935-e7fbc69b954d None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:49.235102 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-26d81026-3c5a-4a83-b935-e7fbc69b954d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:49.244698 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1097/2202] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:46:49.247635 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d81026-3c5a-4a83-b935-e7fbc69b954d None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:49.256064 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c01ae-d44e-4f38-af5c-f9e559df820d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:49.257207 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c01ae-d44e-4f38-af5c-f9e559df820d service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:49.257835 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c01ae-d44e-4f38-af5c-f9e559df820d service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:49.258271 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c01ae-d44e-4f38-af5c-f9e559df820d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:49.270966 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730c01ae-d44e-4f38-af5c-f9e559df820d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:49.272561 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730c01ae-d44e-4f38-af5c-f9e559df820d service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:49.287548 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1106/2203] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:49.361252 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:49.904254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a2d11-194b-4303-ac7f-f83f36d73f48 service neutron] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:49.907758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-2a1a2d11-194b-4303-ac7f-f83f36d73f48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:49.927430 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a2d11-194b-4303-ac7f-f83f36d73f48 service neutron] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:49.933571 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501279e7-f682-42c1-a927-264ee48a47ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:49.934313 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501279e7-f682-42c1-a927-264ee48a47ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:49.934501 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501279e7-f682-42c1-a927-264ee48a47ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:49.934881 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501279e7-f682-42c1-a927-264ee48a47ed admin admin] PATH_INFO: `/v3/projects/ab6e0f4a9c7b466abad562f1c13262c0` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:49.936023 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501279e7-f682-42c1-a927-264ee48a47ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab6e0f4a9c7b466abad562f1c13262c0)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:49.939514 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501279e7-f682-42c1-a927-264ee48a47ed admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:49.956928 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c01ae-d44e-4f38-af5c-f9e559df820d service neutron] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:49.959478 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-730c01ae-d44e-4f38-af5c-f9e559df820d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:49.974619 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:49.979897 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c01ae-d44e-4f38-af5c-f9e559df820d service neutron] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:49.986664 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5e927-a266-4f14-9310-48e564d4544f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:49.987672 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5e927-a266-4f14-9310-48e564d4544f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:49.988113 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5e927-a266-4f14-9310-48e564d4544f admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:49.988671 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5e927-a266-4f14-9310-48e564d4544f admin admin] PATH_INFO: `/v3/projects/fcd2a21b6e6b4bbba070872967d3ded2` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:49.989871 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d5e927-a266-4f14-9310-48e564d4544f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcd2a21b6e6b4bbba070872967d3ded2)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:49.991891 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d5e927-a266-4f14-9310-48e564d4544f admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:50.069588 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-501279e7-f682-42c1-a927-264ee48a47ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab6e0f4a9c7b466abad562f1c13262c0', 'request_id': 'req-501279e7-f682-42c1-a927-264ee48a47ed'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:50.075783 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-501279e7-f682-42c1-a927-264ee48a47ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab6e0f4a9c7b466abad562f1c13262c0', 'request_id': 'req-501279e7-f682-42c1-a927-264ee48a47ed'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:50.144906 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-18d5e927-a266-4f14-9310-48e564d4544f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcd2a21b6e6b4bbba070872967d3ded2', 'request_id': 'req-18d5e927-a266-4f14-9310-48e564d4544f'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:50.154740 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-18d5e927-a266-4f14-9310-48e564d4544f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcd2a21b6e6b4bbba070872967d3ded2', 'request_id': 'req-18d5e927-a266-4f14-9310-48e564d4544f'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:50.182544 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-501279e7-f682-42c1-a927-264ee48a47ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab6e0f4a9c7b466abad562f1c13262c0', 'request_id': 'req-501279e7-f682-42c1-a927-264ee48a47ed'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:50.187641 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:50.195521 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-501279e7-f682-42c1-a927-264ee48a47ed admin admin] The token cache is being invalidate because project ab6e0f4a9c7b466abad562f1c13262c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:50.195818 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-501279e7-f682-42c1-a927-264ee48a47ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:50.196614 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-501279e7-f682-42c1-a927-264ee48a47ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:50.198295 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1098/2204] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:49 2022] DELETE /identity/v3/projects/ab6e0f4a9c7b466abad562f1c13262c0 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:50.202944 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501279e7-f682-42c1-a927-264ee48a47ed admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:50.206496 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-501279e7-f682-42c1-a927-264ee48a47ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:50.296685 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-18d5e927-a266-4f14-9310-48e564d4544f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcd2a21b6e6b4bbba070872967d3ded2', 'request_id': 'req-18d5e927-a266-4f14-9310-48e564d4544f'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:50.304108 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501279e7-f682-42c1-a927-264ee48a47ed admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:50.313110 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-18d5e927-a266-4f14-9310-48e564d4544f admin admin] The token cache is being invalidate because project fcd2a21b6e6b4bbba070872967d3ded2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:50.313810 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-18d5e927-a266-4f14-9310-48e564d4544f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:50.314651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-18d5e927-a266-4f14-9310-48e564d4544f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:50.316345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1107/2205] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:49 2022] DELETE /identity/v3/projects/fcd2a21b6e6b4bbba070872967d3ded2 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:50.321920 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a2918-1056-49ab-8bb8-996792a0cf19 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:50.322609 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a2918-1056-49ab-8bb8-996792a0cf19 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:50.323099 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a2918-1056-49ab-8bb8-996792a0cf19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:50.351975 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-269a2918-1056-49ab-8bb8-996792a0cf19 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:50.352934 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-269a2918-1056-49ab-8bb8-996792a0cf19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:50.382074 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a116d35d-27f9-407e-8bb4-cac9380df9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:50.382804 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116d35d-27f9-407e-8bb4-cac9380df9e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:50.383170 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116d35d-27f9-407e-8bb4-cac9380df9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:50.383526 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116d35d-27f9-407e-8bb4-cac9380df9e4 admin admin] PATH_INFO: `/v3/users/880af75de0cf4818bcf9281ff3059f85` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:50.385195 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a116d35d-27f9-407e-8bb4-cac9380df9e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=880af75de0cf4818bcf9281ff3059f85)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:50.387745 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a116d35d-27f9-407e-8bb4-cac9380df9e4 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:50.432642 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1108/2206] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:46:50.442146 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30303f91-7a93-4af5-9ced-ad1cce3ec910 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:50.442566 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30303f91-7a93-4af5-9ced-ad1cce3ec910 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:50.443033 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30303f91-7a93-4af5-9ced-ad1cce3ec910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:50.465395 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-30303f91-7a93-4af5-9ced-ad1cce3ec910 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:50.466634 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-30303f91-7a93-4af5-9ced-ad1cce3ec910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:50.476004 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1109/2207] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:46:50.480635 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30303f91-7a93-4af5-9ced-ad1cce3ec910 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:50.490525 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30303f91-7a93-4af5-9ced-ad1cce3ec910 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:50.495885 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb96cb-501b-4b91-a455-45dc32d136ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:50.496498 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb96cb-501b-4b91-a455-45dc32d136ec admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:50.496928 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb96cb-501b-4b91-a455-45dc32d136ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:50.497212 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb96cb-501b-4b91-a455-45dc32d136ec admin admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:50.497884 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cb96cb-501b-4b91-a455-45dc32d136ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:46:50.508820 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cb96cb-501b-4b91-a455-45dc32d136ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:50.508820 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cb96cb-501b-4b91-a455-45dc32d136ec admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:50.514147 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1110/2208] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:46:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:46:50.519209 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb96cb-501b-4b91-a455-45dc32d136ec admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:50.529935 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb96cb-501b-4b91-a455-45dc32d136ec admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:50.535265 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d30714-5695-489d-aae8-29e23e1c0a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:50.536172 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d30714-5695-489d-aae8-29e23e1c0a30 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:50.536493 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d30714-5695-489d-aae8-29e23e1c0a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:50.536770 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d30714-5695-489d-aae8-29e23e1c0a30 admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:50.537422 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d30714-5695-489d-aae8-29e23e1c0a30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:50.539361 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d30714-5695-489d-aae8-29e23e1c0a30 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:50.703355 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-a116d35d-27f9-407e-8bb4-cac9380df9e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '880af75de0cf4818bcf9281ff3059f85'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:50.715439 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-a116d35d-27f9-407e-8bb4-cac9380df9e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '880af75de0cf4818bcf9281ff3059f85'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:50.717210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-a116d35d-27f9-407e-8bb4-cac9380df9e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '880af75de0cf4818bcf9281ff3059f85'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:50.735133 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1111/2209] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:50 2022] POST /identity/v3/projects => generated 371 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:46:50.740038 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d30714-5695-489d-aae8-29e23e1c0a30 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:50.742971 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-70d30714-5695-489d-aae8-29e23e1c0a30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:50.811052 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-a116d35d-27f9-407e-8bb4-cac9380df9e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '880af75de0cf4818bcf9281ff3059f85'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:50.812389 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:50.834158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d30714-5695-489d-aae8-29e23e1c0a30 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:50.840775 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-a116d35d-27f9-407e-8bb4-cac9380df9e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '880af75de0cf4818bcf9281ff3059f85'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:50.841097 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35501efe-51ca-443d-a37e-2f80ebf02b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:50.841643 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35501efe-51ca-443d-a37e-2f80ebf02b40 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:50.841797 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35501efe-51ca-443d-a37e-2f80ebf02b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:50.842093 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35501efe-51ca-443d-a37e-2f80ebf02b40 admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:50.842722 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35501efe-51ca-443d-a37e-2f80ebf02b40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:50.845121 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35501efe-51ca-443d-a37e-2f80ebf02b40 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:50.851739 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1099/2210] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:50 2022] DELETE /identity/v3/users/880af75de0cf4818bcf9281ff3059f85 => generated 0 bytes in 650 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:46:50.976446 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1112/2211] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:50 2022] POST /identity/v3/users => generated 286 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:46:50.977213 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a116d35d-27f9-407e-8bb4-cac9380df9e4 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:51.002893 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:51.049810 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a116d35d-27f9-407e-8bb4-cac9380df9e4 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:51.054571 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c322d662-bc7d-4410-8219-3ee33484c689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:51.055186 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c322d662-bc7d-4410-8219-3ee33484c689 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:51.055471 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c322d662-bc7d-4410-8219-3ee33484c689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:51.055747 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c322d662-bc7d-4410-8219-3ee33484c689 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:51.056427 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c322d662-bc7d-4410-8219-3ee33484c689 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:51.058094 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c322d662-bc7d-4410-8219-3ee33484c689 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:51.068813 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1100/2212] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:50 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:46:51.072988 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35501efe-51ca-443d-a37e-2f80ebf02b40 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:51.089040 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35501efe-51ca-443d-a37e-2f80ebf02b40 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:51.093908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117e31b-f3e1-47e2-b41d-2bad05643691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:51.094528 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117e31b-f3e1-47e2-b41d-2bad05643691 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:51.095350 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117e31b-f3e1-47e2-b41d-2bad05643691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:51.095624 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117e31b-f3e1-47e2-b41d-2bad05643691 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:51.096517 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2117e31b-f3e1-47e2-b41d-2bad05643691 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:51.098162 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2117e31b-f3e1-47e2-b41d-2bad05643691 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:51.109225 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1113/2213] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:51.115092 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c322d662-bc7d-4410-8219-3ee33484c689 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:51.128155 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c322d662-bc7d-4410-8219-3ee33484c689 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:51.134541 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e748c-5b56-4480-9b1b-c7cda09b97a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:51.138068 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e748c-5b56-4480-9b1b-c7cda09b97a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:51.138068 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e748c-5b56-4480-9b1b-c7cda09b97a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:51.138068 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e748c-5b56-4480-9b1b-c7cda09b97a4 admin admin] PATH_INFO: `/v3/projects/efcbca3e4ee149b0bed5ea61c3a42755/users/9902d63688e24ead8efb75d2fcdbdc14/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:51.155560 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461e748c-5b56-4480-9b1b-c7cda09b97a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efcbca3e4ee149b0bed5ea61c3a42755, user_id=9902d63688e24ead8efb75d2fcdbdc14, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:51.163913 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461e748c-5b56-4480-9b1b-c7cda09b97a4 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:51.277656 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1101/2214] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:51 2022] PUT /identity/v3/projects/efcbca3e4ee149b0bed5ea61c3a42755/users/9902d63688e24ead8efb75d2fcdbdc14/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:46:51.295284 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0700d2-df89-41e0-9b10-b2d08a1a7506 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:51.295694 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0700d2-df89-41e0-9b10-b2d08a1a7506 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:51.295979 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0700d2-df89-41e0-9b10-b2d08a1a7506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:51.331333 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-0a0700d2-df89-41e0-9b10-b2d08a1a7506 None None] MFA Rules not processed for user `9902d63688e24ead8efb75d2fcdbdc14`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:51.333067 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-0a0700d2-df89-41e0-9b10-b2d08a1a7506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:51.401366 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e748c-5b56-4480-9b1b-c7cda09b97a4 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:51.426584 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-461e748c-5b56-4480-9b1b-c7cda09b97a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:51.429877 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1114/2215] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:51.446182 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0700d2-df89-41e0-9b10-b2d08a1a7506 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:51.449863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-0a0700d2-df89-41e0-9b10-b2d08a1a7506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:51.522417 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e748c-5b56-4480-9b1b-c7cda09b97a4 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:51.524110 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0700d2-df89-41e0-9b10-b2d08a1a7506 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:51.528440 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94faad6e-d423-4a1b-b53e-fec2221f2a3c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9aee9c524a974a248c3bf363444f584a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:51.528776 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa6eea-2895-434d-9d8c-3b2db8452e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:51.528980 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94faad6e-d423-4a1b-b53e-fec2221f2a3c service glance] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:51.529170 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa6eea-2895-434d-9d8c-3b2db8452e25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:51.529376 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94faad6e-d423-4a1b-b53e-fec2221f2a3c service glance] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:51.529747 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94faad6e-d423-4a1b-b53e-fec2221f2a3c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:51.529936 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa6eea-2895-434d-9d8c-3b2db8452e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:51.530355 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa6eea-2895-434d-9d8c-3b2db8452e25 admin admin] PATH_INFO: `/v3/projects/e869874ba4e74ef39b3143b5bb187d00` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:51.533133 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fa6eea-2895-434d-9d8c-3b2db8452e25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e869874ba4e74ef39b3143b5bb187d00)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:51.535881 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fa6eea-2895-434d-9d8c-3b2db8452e25 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:51.541538 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94faad6e-d423-4a1b-b53e-fec2221f2a3c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:51.543691 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94faad6e-d423-4a1b-b53e-fec2221f2a3c service glance] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:51.555915 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1115/2216] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:46:51.669592 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-56fa6eea-2895-434d-9d8c-3b2db8452e25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e869874ba4e74ef39b3143b5bb187d00', 'request_id': 'req-56fa6eea-2895-434d-9d8c-3b2db8452e25'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:51.682264 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-56fa6eea-2895-434d-9d8c-3b2db8452e25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e869874ba4e74ef39b3143b5bb187d00', 'request_id': 'req-56fa6eea-2895-434d-9d8c-3b2db8452e25'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:51.784032 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-56fa6eea-2895-434d-9d8c-3b2db8452e25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e869874ba4e74ef39b3143b5bb187d00', 'request_id': 'req-56fa6eea-2895-434d-9d8c-3b2db8452e25'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:51.799990 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-56fa6eea-2895-434d-9d8c-3b2db8452e25 admin admin] The token cache is being invalidate because project e869874ba4e74ef39b3143b5bb187d00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:51.800284 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-56fa6eea-2895-434d-9d8c-3b2db8452e25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:51.801132 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-56fa6eea-2895-434d-9d8c-3b2db8452e25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:51.802776 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1102/2217] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:51 2022] DELETE /identity/v3/projects/e869874ba4e74ef39b3143b5bb187d00 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:51.810585 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ac2e0-052f-4531-8541-1cb7f5ca6adf None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:51.810922 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ac2e0-052f-4531-8541-1cb7f5ca6adf None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:51.811366 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ac2e0-052f-4531-8541-1cb7f5ca6adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:51.842867 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-647ac2e0-052f-4531-8541-1cb7f5ca6adf None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:51.844325 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-647ac2e0-052f-4531-8541-1cb7f5ca6adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:51.947206 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:51.962019 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1116/2218] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:46:51.975133 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c39b962-8f23-4a09-800d-ce8e573767e5 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:51.977212 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c39b962-8f23-4a09-800d-ce8e573767e5 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:51.977805 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c39b962-8f23-4a09-800d-ce8e573767e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:52.007449 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-9c39b962-8f23-4a09-800d-ce8e573767e5 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:52.008863 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-9c39b962-8f23-4a09-800d-ce8e573767e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:52.018090 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1103/2219] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:52.024533 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ac2e0-052f-4531-8541-1cb7f5ca6adf None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:52.037913 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ac2e0-052f-4531-8541-1cb7f5ca6adf None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:52.044594 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bddd031-d719-4f87-9786-c76292dbc59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:52.045604 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bddd031-d719-4f87-9786-c76292dbc59d admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:52.045714 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bddd031-d719-4f87-9786-c76292dbc59d admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:52.045942 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bddd031-d719-4f87-9786-c76292dbc59d admin admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:52.047555 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bddd031-d719-4f87-9786-c76292dbc59d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:46:52.047555 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bddd031-d719-4f87-9786-c76292dbc59d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:52.050100 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bddd031-d719-4f87-9786-c76292dbc59d admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:52.071348 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1117/2220] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:46:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:46:52.076372 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c39b962-8f23-4a09-800d-ce8e573767e5 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:52.092282 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c39b962-8f23-4a09-800d-ce8e573767e5 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:52.105253 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d381c3-5dcb-4353-a05c-68fdd758a956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:52.106477 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d381c3-5dcb-4353-a05c-68fdd758a956 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:52.106843 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d381c3-5dcb-4353-a05c-68fdd758a956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:52.107261 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d381c3-5dcb-4353-a05c-68fdd758a956 admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:52.108078 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d381c3-5dcb-4353-a05c-68fdd758a956 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:52.111227 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d381c3-5dcb-4353-a05c-68fdd758a956 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:52.279779 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1104/2221] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:52 2022] POST /identity/v3/projects => generated 379 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:52.285769 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bddd031-d719-4f87-9786-c76292dbc59d admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:52.301925 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bddd031-d719-4f87-9786-c76292dbc59d admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:52.352163 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:52.378275 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0063277-8224-427b-8d67-c12fcfc4cbe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:52.379014 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0063277-8224-427b-8d67-c12fcfc4cbe2 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:52.379165 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0063277-8224-427b-8d67-c12fcfc4cbe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:52.379435 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0063277-8224-427b-8d67-c12fcfc4cbe2 admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:52.380077 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0063277-8224-427b-8d67-c12fcfc4cbe2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:52.382163 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0063277-8224-427b-8d67-c12fcfc4cbe2 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:52.504385 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1118/2222] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:52 2022] POST /identity/v3/users => generated 290 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:52.509157 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d381c3-5dcb-4353-a05c-68fdd758a956 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:52.522200 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d381c3-5dcb-4353-a05c-68fdd758a956 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:52.527726 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0ef366-66b7-4e21-a7c7-94078dfbc17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:52.528957 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ef366-66b7-4e21-a7c7-94078dfbc17f admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:52.529198 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ef366-66b7-4e21-a7c7-94078dfbc17f admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:52.529664 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ef366-66b7-4e21-a7c7-94078dfbc17f admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:52.531186 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0ef366-66b7-4e21-a7c7-94078dfbc17f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:52.534526 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0ef366-66b7-4e21-a7c7-94078dfbc17f admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:52.555664 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1105/2223] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:52 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:52.560487 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0063277-8224-427b-8d67-c12fcfc4cbe2 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:52.576909 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0063277-8224-427b-8d67-c12fcfc4cbe2 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:52.583535 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be91a56-a125-4609-aa8f-4f325c07ae51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:52.584507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be91a56-a125-4609-aa8f-4f325c07ae51 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:52.584802 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be91a56-a125-4609-aa8f-4f325c07ae51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:52.585112 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be91a56-a125-4609-aa8f-4f325c07ae51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:52.585962 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be91a56-a125-4609-aa8f-4f325c07ae51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:52.589253 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be91a56-a125-4609-aa8f-4f325c07ae51 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:52.611688 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1119/2224] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:52 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:52.616442 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0ef366-66b7-4e21-a7c7-94078dfbc17f admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:52.632773 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0ef366-66b7-4e21-a7c7-94078dfbc17f admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:52.641029 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d0a98-91f9-4dd2-8abf-be3161641343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:52.641918 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d0a98-91f9-4dd2-8abf-be3161641343 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:52.642201 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d0a98-91f9-4dd2-8abf-be3161641343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:52.642482 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d0a98-91f9-4dd2-8abf-be3161641343 admin admin] PATH_INFO: `/v3/projects/64895d8cbe00459f8e3ac08cab8d1147/users/ed8b456c99664f42b95b0c13e12a3585/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:52.695719 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838d0a98-91f9-4dd2-8abf-be3161641343 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64895d8cbe00459f8e3ac08cab8d1147, user_id=ed8b456c99664f42b95b0c13e12a3585, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:52.699094 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838d0a98-91f9-4dd2-8abf-be3161641343 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:52.794031 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1106/2225] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:52 2022] PUT /identity/v3/projects/64895d8cbe00459f8e3ac08cab8d1147/users/ed8b456c99664f42b95b0c13e12a3585/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:52.807086 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9beda-3ada-40aa-a20f-e14419c07e5f None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:52.807486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9beda-3ada-40aa-a20f-e14419c07e5f None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:52.807979 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9beda-3ada-40aa-a20f-e14419c07e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:52.874200 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-76d9beda-3ada-40aa-a20f-e14419c07e5f None None] MFA Rules not processed for user `ed8b456c99664f42b95b0c13e12a3585`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:52.875634 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-76d9beda-3ada-40aa-a20f-e14419c07e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:52.985297 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1120/2226] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:52.993502 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d0a98-91f9-4dd2-8abf-be3161641343 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:53.131727 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d0a98-91f9-4dd2-8abf-be3161641343 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:53.136888 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d12b84-07f2-42c5-826d-fd1790e8609a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:53.137816 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d12b84-07f2-42c5-826d-fd1790e8609a admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:53.138142 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d12b84-07f2-42c5-826d-fd1790e8609a admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:53.138426 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d12b84-07f2-42c5-826d-fd1790e8609a admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:53.139163 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d12b84-07f2-42c5-826d-fd1790e8609a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:53.141286 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d12b84-07f2-42c5-826d-fd1790e8609a admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:53.325072 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1107/2227] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:52 2022] POST /identity/v3/projects => generated 379 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:53.326252 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d9beda-3ada-40aa-a20f-e14419c07e5f None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:53.340909 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d9beda-3ada-40aa-a20f-e14419c07e5f None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:53.403893 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:53.417688 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30205ea7-9df1-42a1-b20a-f438b70a4c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:53.418330 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30205ea7-9df1-42a1-b20a-f438b70a4c3a admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:53.418611 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30205ea7-9df1-42a1-b20a-f438b70a4c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:53.418896 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30205ea7-9df1-42a1-b20a-f438b70a4c3a admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:53.419576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30205ea7-9df1-42a1-b20a-f438b70a4c3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:53.421688 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30205ea7-9df1-42a1-b20a-f438b70a4c3a admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:53.555394 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1121/2228] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:53 2022] POST /identity/v3/users => generated 296 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:53.556669 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d12b84-07f2-42c5-826d-fd1790e8609a admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:53.579987 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d12b84-07f2-42c5-826d-fd1790e8609a admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:53.588771 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d05d71-23f9-44cc-8996-9734f23d51c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:53.589856 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d05d71-23f9-44cc-8996-9734f23d51c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:53.590390 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d05d71-23f9-44cc-8996-9734f23d51c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:53.590848 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d05d71-23f9-44cc-8996-9734f23d51c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:53.591908 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d05d71-23f9-44cc-8996-9734f23d51c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:53.598258 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d05d71-23f9-44cc-8996-9734f23d51c3 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:53.614045 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1108/2229] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:53 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:53.619832 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30205ea7-9df1-42a1-b20a-f438b70a4c3a admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:53.631159 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30205ea7-9df1-42a1-b20a-f438b70a4c3a admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:53.638963 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f76a7a2-c3cf-4863-9295-b1754c40e2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:53.640138 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76a7a2-c3cf-4863-9295-b1754c40e2f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:53.640725 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76a7a2-c3cf-4863-9295-b1754c40e2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:53.641229 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76a7a2-c3cf-4863-9295-b1754c40e2f2 admin admin] PATH_INFO: `/v3/projects/871bbc585ee045b290c4dcc982ed1011/users/70c1a287b84b4276a35ce159cadeaeaf/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:53.670485 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f76a7a2-c3cf-4863-9295-b1754c40e2f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=871bbc585ee045b290c4dcc982ed1011, user_id=70c1a287b84b4276a35ce159cadeaeaf, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:53.676684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f76a7a2-c3cf-4863-9295-b1754c40e2f2 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:53.772234 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1122/2230] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:53 2022] PUT /identity/v3/projects/871bbc585ee045b290c4dcc982ed1011/users/70c1a287b84b4276a35ce159cadeaeaf/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:53.776430 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d05d71-23f9-44cc-8996-9734f23d51c3 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:53.792274 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d05d71-23f9-44cc-8996-9734f23d51c3 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:53.859656 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b503b6-5c42-48f0-899a-7cc3c398d87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:53.860206 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b503b6-5c42-48f0-899a-7cc3c398d87d admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:53.861268 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b503b6-5c42-48f0-899a-7cc3c398d87d admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:53.861658 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b503b6-5c42-48f0-899a-7cc3c398d87d admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:53.864509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b503b6-5c42-48f0-899a-7cc3c398d87d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:53.864509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b503b6-5c42-48f0-899a-7cc3c398d87d admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:53.876772 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1109/2231] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:53 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:53.881490 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f76a7a2-c3cf-4863-9295-b1754c40e2f2 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:53.897750 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f76a7a2-c3cf-4863-9295-b1754c40e2f2 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:53.903677 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5131f3f-2d45-4034-b597-f1ae86ce3e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:53.904200 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5131f3f-2d45-4034-b597-f1ae86ce3e16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:53.904200 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5131f3f-2d45-4034-b597-f1ae86ce3e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:53.908490 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5131f3f-2d45-4034-b597-f1ae86ce3e16 admin admin] PATH_INFO: `/v3/projects/871bbc585ee045b290c4dcc982ed1011/users/70c1a287b84b4276a35ce159cadeaeaf/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:53.908490 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5131f3f-2d45-4034-b597-f1ae86ce3e16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=871bbc585ee045b290c4dcc982ed1011, user_id=70c1a287b84b4276a35ce159cadeaeaf, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:53.910158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5131f3f-2d45-4034-b597-f1ae86ce3e16 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:53.922708 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1123/2232] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:53 2022] PUT /identity/v3/projects/871bbc585ee045b290c4dcc982ed1011/users/70c1a287b84b4276a35ce159cadeaeaf/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:46:53.935622 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3950a8-9c13-4db1-9710-986a6df32c28 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:53.937365 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3950a8-9c13-4db1-9710-986a6df32c28 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:53.938737 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3950a8-9c13-4db1-9710-986a6df32c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:53.972874 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-be3950a8-9c13-4db1-9710-986a6df32c28 None None] MFA Rules not processed for user `70c1a287b84b4276a35ce159cadeaeaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:53.974118 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-be3950a8-9c13-4db1-9710-986a6df32c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:53.995162 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:54.075874 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1110/2233] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:53 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:54.091219 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5131f3f-2d45-4034-b597-f1ae86ce3e16 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:54.093130 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-e5131f3f-2d45-4034-b597-f1ae86ce3e16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:54.166706 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5131f3f-2d45-4034-b597-f1ae86ce3e16 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:54.170946 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f814d3-7c78-4b6c-b827-3093a0f46a5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:54.171573 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f814d3-7c78-4b6c-b827-3093a0f46a5a service cinder] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:54.171885 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f814d3-7c78-4b6c-b827-3093a0f46a5a service cinder] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:54.172159 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f814d3-7c78-4b6c-b827-3093a0f46a5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:54.190058 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f814d3-7c78-4b6c-b827-3093a0f46a5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:54.192559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f814d3-7c78-4b6c-b827-3093a0f46a5a service cinder] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:54.204054 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1124/2234] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:54 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:55.028347 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3950a8-9c13-4db1-9710-986a6df32c28 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:55.129971 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3950a8-9c13-4db1-9710-986a6df32c28 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:55.141996 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791dfba0-9198-46d5-a950-63d2f8cb7e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:55.141996 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791dfba0-9198-46d5-a950-63d2f8cb7e22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:55.141996 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791dfba0-9198-46d5-a950-63d2f8cb7e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:55.141996 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791dfba0-9198-46d5-a950-63d2f8cb7e22 admin admin] PATH_INFO: `/v3/users/ed8b456c99664f42b95b0c13e12a3585` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:55.143667 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791dfba0-9198-46d5-a950-63d2f8cb7e22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed8b456c99664f42b95b0c13e12a3585)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:55.148241 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791dfba0-9198-46d5-a950-63d2f8cb7e22 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:55.460830 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-791dfba0-9198-46d5-a950-63d2f8cb7e22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed8b456c99664f42b95b0c13e12a3585'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:55.469326 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-791dfba0-9198-46d5-a950-63d2f8cb7e22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed8b456c99664f42b95b0c13e12a3585'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:55.470392 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-791dfba0-9198-46d5-a950-63d2f8cb7e22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed8b456c99664f42b95b0c13e12a3585'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:55.576906 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-791dfba0-9198-46d5-a950-63d2f8cb7e22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed8b456c99664f42b95b0c13e12a3585'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:55.593420 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-791dfba0-9198-46d5-a950-63d2f8cb7e22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed8b456c99664f42b95b0c13e12a3585'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:55.600604 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1111/2235] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:55 2022] DELETE /identity/v3/users/ed8b456c99664f42b95b0c13e12a3585 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:55.606536 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392ffe1-02f8-45f9-96df-b300d8f6213d None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:55.607117 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392ffe1-02f8-45f9-96df-b300d8f6213d None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:55.607523 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392ffe1-02f8-45f9-96df-b300d8f6213d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:55.633595 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-a392ffe1-02f8-45f9-96df-b300d8f6213d None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:55.635741 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-a392ffe1-02f8-45f9-96df-b300d8f6213d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:55.743359 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1125/2236] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:55.754342 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791dfba0-9198-46d5-a950-63d2f8cb7e22 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:55.755796 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-791dfba0-9198-46d5-a950-63d2f8cb7e22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:55.795668 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:55.823310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791dfba0-9198-46d5-a950-63d2f8cb7e22 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:55.827421 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a3cbd3-526a-4c17-be20-7536eb106c82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:55.827974 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3cbd3-526a-4c17-be20-7536eb106c82 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:55.828259 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3cbd3-526a-4c17-be20-7536eb106c82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:55.828592 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3cbd3-526a-4c17-be20-7536eb106c82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:55.839742 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a3cbd3-526a-4c17-be20-7536eb106c82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:55.841340 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a3cbd3-526a-4c17-be20-7536eb106c82 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:55.852738 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1112/2237] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:46:56.608984 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392ffe1-02f8-45f9-96df-b300d8f6213d None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:56.611473 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-a392ffe1-02f8-45f9-96df-b300d8f6213d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:56.637617 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392ffe1-02f8-45f9-96df-b300d8f6213d None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:56.646594 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f938b16f-6139-42ca-93a6-53189d604e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:56.647503 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f938b16f-6139-42ca-93a6-53189d604e8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:56.647850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f938b16f-6139-42ca-93a6-53189d604e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:56.648286 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f938b16f-6139-42ca-93a6-53189d604e8f admin admin] PATH_INFO: `/v3/projects/64895d8cbe00459f8e3ac08cab8d1147` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:56.650346 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f938b16f-6139-42ca-93a6-53189d604e8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=64895d8cbe00459f8e3ac08cab8d1147)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:56.652444 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f938b16f-6139-42ca-93a6-53189d604e8f admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:56.790400 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:56.795455 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-f938b16f-6139-42ca-93a6-53189d604e8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64895d8cbe00459f8e3ac08cab8d1147', 'request_id': 'req-f938b16f-6139-42ca-93a6-53189d604e8f'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:56.803548 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-f938b16f-6139-42ca-93a6-53189d604e8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64895d8cbe00459f8e3ac08cab8d1147', 'request_id': 'req-f938b16f-6139-42ca-93a6-53189d604e8f'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:56.920061 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-f938b16f-6139-42ca-93a6-53189d604e8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64895d8cbe00459f8e3ac08cab8d1147', 'request_id': 'req-f938b16f-6139-42ca-93a6-53189d604e8f'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:56.936353 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-f938b16f-6139-42ca-93a6-53189d604e8f admin admin] The token cache is being invalidate because project 64895d8cbe00459f8e3ac08cab8d1147 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:56.936681 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-f938b16f-6139-42ca-93a6-53189d604e8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:56.937358 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-f938b16f-6139-42ca-93a6-53189d604e8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:56.939214 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1126/2238] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:56 2022] DELETE /identity/v3/projects/64895d8cbe00459f8e3ac08cab8d1147 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:56.944039 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a3cbd3-526a-4c17-be20-7536eb106c82 service neutron] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:56.956241 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a3cbd3-526a-4c17-be20-7536eb106c82 service neutron] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:57.037930 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0103ddf7-153b-422a-ab09-e828579f1171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:57.038778 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103ddf7-153b-422a-ab09-e828579f1171 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:57.039667 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103ddf7-153b-422a-ab09-e828579f1171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:57.039667 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103ddf7-153b-422a-ab09-e828579f1171 admin admin] PATH_INFO: `/v3/users/70c1a287b84b4276a35ce159cadeaeaf` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:57.040595 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0103ddf7-153b-422a-ab09-e828579f1171 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70c1a287b84b4276a35ce159cadeaeaf)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:57.043197 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0103ddf7-153b-422a-ab09-e828579f1171 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:57.407759 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-0103ddf7-153b-422a-ab09-e828579f1171 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70c1a287b84b4276a35ce159cadeaeaf'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:57.430267 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-0103ddf7-153b-422a-ab09-e828579f1171 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70c1a287b84b4276a35ce159cadeaeaf'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:57.431371 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-0103ddf7-153b-422a-ab09-e828579f1171 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70c1a287b84b4276a35ce159cadeaeaf'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:57.605380 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-0103ddf7-153b-422a-ab09-e828579f1171 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70c1a287b84b4276a35ce159cadeaeaf'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:57.625068 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-0103ddf7-153b-422a-ab09-e828579f1171 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70c1a287b84b4276a35ce159cadeaeaf'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:57.633242 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1113/2239] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:56 2022] DELETE /identity/v3/users/70c1a287b84b4276a35ce159cadeaeaf => generated 0 bytes in 690 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:46:58.104471 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f938b16f-6139-42ca-93a6-53189d604e8f admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:58.106052 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-f938b16f-6139-42ca-93a6-53189d604e8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:58.133186 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f938b16f-6139-42ca-93a6-53189d604e8f admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:58.235647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b9519c-adc1-4744-a1c6-35a55cb59297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:58.236750 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9519c-adc1-4744-a1c6-35a55cb59297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:58.237176 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9519c-adc1-4744-a1c6-35a55cb59297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:58.237530 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9519c-adc1-4744-a1c6-35a55cb59297 admin admin] PATH_INFO: `/v3/projects/871bbc585ee045b290c4dcc982ed1011` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:58.239018 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b9519c-adc1-4744-a1c6-35a55cb59297 admin admin] RBAC: Authorizing `identity:delete_project(project_id=871bbc585ee045b290c4dcc982ed1011)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:58.241261 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b9519c-adc1-4744-a1c6-35a55cb59297 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:58.385092 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-c6b9519c-adc1-4744-a1c6-35a55cb59297 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '871bbc585ee045b290c4dcc982ed1011', 'request_id': 'req-c6b9519c-adc1-4744-a1c6-35a55cb59297'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:58.393537 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-c6b9519c-adc1-4744-a1c6-35a55cb59297 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '871bbc585ee045b290c4dcc982ed1011', 'request_id': 'req-c6b9519c-adc1-4744-a1c6-35a55cb59297'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:58.530913 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-c6b9519c-adc1-4744-a1c6-35a55cb59297 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '871bbc585ee045b290c4dcc982ed1011', 'request_id': 'req-c6b9519c-adc1-4744-a1c6-35a55cb59297'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:58.543803 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-c6b9519c-adc1-4744-a1c6-35a55cb59297 admin admin] The token cache is being invalidate because project 871bbc585ee045b290c4dcc982ed1011 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:58.544341 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-c6b9519c-adc1-4744-a1c6-35a55cb59297 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:58.545433 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-c6b9519c-adc1-4744-a1c6-35a55cb59297 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:58.547213 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1127/2240] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:58 2022] DELETE /identity/v3/projects/871bbc585ee045b290c4dcc982ed1011 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:58.553967 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c13229c-26aa-44b8-980a-1e00f1aac7aa None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:58.554567 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c13229c-26aa-44b8-980a-1e00f1aac7aa None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:58.555058 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c13229c-26aa-44b8-980a-1e00f1aac7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:58.581628 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-4c13229c-26aa-44b8-980a-1e00f1aac7aa None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:58.582928 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-4c13229c-26aa-44b8-980a-1e00f1aac7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:58.679560 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:58.697322 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1114/2241] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:46:58.706558 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94006581-2d99-4393-9bfd-eda47f9306ba None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:58.706861 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94006581-2d99-4393-9bfd-eda47f9306ba None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:58.707155 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94006581-2d99-4393-9bfd-eda47f9306ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:58.730275 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-94006581-2d99-4393-9bfd-eda47f9306ba None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:58.731952 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-94006581-2d99-4393-9bfd-eda47f9306ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:58.741901 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1128/2242] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:58.746828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c13229c-26aa-44b8-980a-1e00f1aac7aa None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:58.787334 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c13229c-26aa-44b8-980a-1e00f1aac7aa None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:58.797578 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f2c66-483d-4f06-beb8-ec6f0f0590c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:58.799511 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f2c66-483d-4f06-beb8-ec6f0f0590c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:58.800103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f2c66-483d-4f06-beb8-ec6f0f0590c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:58.800729 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f2c66-483d-4f06-beb8-ec6f0f0590c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:58.805567 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395f2c66-483d-4f06-beb8-ec6f0f0590c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:46:58.805567 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395f2c66-483d-4f06-beb8-ec6f0f0590c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:58.807429 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395f2c66-483d-4f06-beb8-ec6f0f0590c4 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:58.840256 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1115/2243] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:46:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:46:58.846431 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94006581-2d99-4393-9bfd-eda47f9306ba None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:58.866211 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94006581-2d99-4393-9bfd-eda47f9306ba None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:58.876028 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba40233-d6f2-410e-9914-1aeb97c829af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:58.878585 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba40233-d6f2-410e-9914-1aeb97c829af admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:58.879105 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba40233-d6f2-410e-9914-1aeb97c829af admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:58.880310 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba40233-d6f2-410e-9914-1aeb97c829af admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:58.881611 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba40233-d6f2-410e-9914-1aeb97c829af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:58.886066 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba40233-d6f2-410e-9914-1aeb97c829af admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:59.069645 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1129/2244] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:58 2022] POST /identity/v3/projects => generated 375 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:59.071681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f2c66-483d-4f06-beb8-ec6f0f0590c4 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:59.084183 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f2c66-483d-4f06-beb8-ec6f0f0590c4 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:59.186758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223c4370-5156-42ab-a9c2-8c6e55fbbaff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:59.187687 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c4370-5156-42ab-a9c2-8c6e55fbbaff admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:59.188073 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c4370-5156-42ab-a9c2-8c6e55fbbaff admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:59.188476 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c4370-5156-42ab-a9c2-8c6e55fbbaff admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:59.189320 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223c4370-5156-42ab-a9c2-8c6e55fbbaff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:59.191615 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223c4370-5156-42ab-a9c2-8c6e55fbbaff admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:59.324134 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1116/2245] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:59 2022] POST /identity/v3/users => generated 288 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:59.329070 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba40233-d6f2-410e-9914-1aeb97c829af admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:59.344611 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba40233-d6f2-410e-9914-1aeb97c829af admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:59.351277 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d95f3ff-a52b-4ff4-a212-a34b3fa94818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:59.352069 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d95f3ff-a52b-4ff4-a212-a34b3fa94818 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:59.352501 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d95f3ff-a52b-4ff4-a212-a34b3fa94818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:59.352866 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d95f3ff-a52b-4ff4-a212-a34b3fa94818 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:59.353808 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d95f3ff-a52b-4ff4-a212-a34b3fa94818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:59.355746 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d95f3ff-a52b-4ff4-a212-a34b3fa94818 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:59.369538 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1130/2246] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:59 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:59.375119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223c4370-5156-42ab-a9c2-8c6e55fbbaff admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:59.391070 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223c4370-5156-42ab-a9c2-8c6e55fbbaff admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:59.398687 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e788323-a54c-4718-aea8-5bf96bfb506e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:59.399713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e788323-a54c-4718-aea8-5bf96bfb506e admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:59.400189 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e788323-a54c-4718-aea8-5bf96bfb506e admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:59.400887 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e788323-a54c-4718-aea8-5bf96bfb506e admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:59.402131 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e788323-a54c-4718-aea8-5bf96bfb506e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:59.405452 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e788323-a54c-4718-aea8-5bf96bfb506e admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:59.425953 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1117/2247] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:59 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:59.430634 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d95f3ff-a52b-4ff4-a212-a34b3fa94818 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:59.441762 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d95f3ff-a52b-4ff4-a212-a34b3fa94818 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:59.446602 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa630ad-f449-4360-afe3-b49ce8215b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:59.447336 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa630ad-f449-4360-afe3-b49ce8215b91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:59.447617 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa630ad-f449-4360-afe3-b49ce8215b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:59.447895 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa630ad-f449-4360-afe3-b49ce8215b91 admin admin] PATH_INFO: `/v3/projects/376bd21a941b4beea5f3c4e98c6c1468/users/27453b6103fc46f28878768474245b24/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:59.464598 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa630ad-f449-4360-afe3-b49ce8215b91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=376bd21a941b4beea5f3c4e98c6c1468, user_id=27453b6103fc46f28878768474245b24, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:59.467484 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa630ad-f449-4360-afe3-b49ce8215b91 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:59.572531 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:59.578784 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1131/2248] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:59 2022] PUT /identity/v3/projects/376bd21a941b4beea5f3c4e98c6c1468/users/27453b6103fc46f28878768474245b24/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:59.589387 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c83540-979d-4e71-a04a-6680919e675f None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:59.589892 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c83540-979d-4e71-a04a-6680919e675f None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:59.590012 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c83540-979d-4e71-a04a-6680919e675f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:59.623877 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-f2c83540-979d-4e71-a04a-6680919e675f None None] MFA Rules not processed for user `27453b6103fc46f28878768474245b24`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:59.625318 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-f2c83540-979d-4e71-a04a-6680919e675f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:59.729366 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1118/2249] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:46:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:59.741884 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa630ad-f449-4360-afe3-b49ce8215b91 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:59.745908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-6fa630ad-f449-4360-afe3-b49ce8215b91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:59.824269 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa630ad-f449-4360-afe3-b49ce8215b91 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:59.828943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2f104b-af9b-4c22-8fba-ba4b7826b39a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9aee9c524a974a248c3bf363444f584a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:59.829675 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f104b-af9b-4c22-8fba-ba4b7826b39a service glance] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:59.829976 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f104b-af9b-4c22-8fba-ba4b7826b39a service glance] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:59.830271 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f104b-af9b-4c22-8fba-ba4b7826b39a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:59.839927 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2f104b-af9b-4c22-8fba-ba4b7826b39a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:59.841601 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2f104b-af9b-4c22-8fba-ba4b7826b39a service glance] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:59.853873 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1132/2250] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:59.972028 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c83540-979d-4e71-a04a-6680919e675f None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:59.974455 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-f2c83540-979d-4e71-a04a-6680919e675f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:00.079065 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:00.102566 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c83540-979d-4e71-a04a-6680919e675f None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:00.112676 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2107818d-8c82-4d60-8164-2e2eb2386a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:00.113523 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107818d-8c82-4d60-8164-2e2eb2386a04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:00.114189 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107818d-8c82-4d60-8164-2e2eb2386a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:00.115309 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107818d-8c82-4d60-8164-2e2eb2386a04 admin admin] PATH_INFO: `/v3/users/e44a0edca1604fb2aee88420a77ab822` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:00.117127 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2107818d-8c82-4d60-8164-2e2eb2386a04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e44a0edca1604fb2aee88420a77ab822)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:00.119707 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2107818d-8c82-4d60-8164-2e2eb2386a04 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:00.478353 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-2107818d-8c82-4d60-8164-2e2eb2386a04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e44a0edca1604fb2aee88420a77ab822'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:00.488633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-2107818d-8c82-4d60-8164-2e2eb2386a04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e44a0edca1604fb2aee88420a77ab822'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:00.489409 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-2107818d-8c82-4d60-8164-2e2eb2386a04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e44a0edca1604fb2aee88420a77ab822'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:00.628900 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-2107818d-8c82-4d60-8164-2e2eb2386a04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e44a0edca1604fb2aee88420a77ab822'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:00.646084 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-2107818d-8c82-4d60-8164-2e2eb2386a04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e44a0edca1604fb2aee88420a77ab822'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:00.652440 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1119/2251] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:59 2022] DELETE /identity/v3/users/e44a0edca1604fb2aee88420a77ab822 => generated 0 bytes in 682 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:00.656964 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d5c11-ff15-4b65-a99a-d847ec1026d0 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:00.658064 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d5c11-ff15-4b65-a99a-d847ec1026d0 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:00.658064 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d5c11-ff15-4b65-a99a-d847ec1026d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:00.678755 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-1d1d5c11-ff15-4b65-a99a-d847ec1026d0 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:00.680000 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-1d1d5c11-ff15-4b65-a99a-d847ec1026d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:00.746255 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1133/2252] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:47:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:47:00.756042 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2107818d-8c82-4d60-8164-2e2eb2386a04 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:00.757998 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-2107818d-8c82-4d60-8164-2e2eb2386a04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:00.879041 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2107818d-8c82-4d60-8164-2e2eb2386a04 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:00.883930 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19392e32-4c78-491d-b551-980099c7f443 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:00.884772 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19392e32-4c78-491d-b551-980099c7f443 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:00.885106 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19392e32-4c78-491d-b551-980099c7f443 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:00.885429 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19392e32-4c78-491d-b551-980099c7f443 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:00.898165 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19392e32-4c78-491d-b551-980099c7f443 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:00.899678 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19392e32-4c78-491d-b551-980099c7f443 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:00.915177 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1120/2253] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:47:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:47:01.828089 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d5c11-ff15-4b65-a99a-d847ec1026d0 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:01.830167 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-1d1d5c11-ff15-4b65-a99a-d847ec1026d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:01.852716 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d5c11-ff15-4b65-a99a-d847ec1026d0 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:01.870162 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1d76d7-3745-40d6-b41f-d7fc09b834b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:01.878060 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d76d7-3745-40d6-b41f-d7fc09b834b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:01.878711 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d76d7-3745-40d6-b41f-d7fc09b834b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:01.879005 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d76d7-3745-40d6-b41f-d7fc09b834b2 admin admin] PATH_INFO: `/v3/projects/1578db0219964206875af31847275d90` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:01.887444 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1d76d7-3745-40d6-b41f-d7fc09b834b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1578db0219964206875af31847275d90)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:01.887444 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1d76d7-3745-40d6-b41f-d7fc09b834b2 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:02.053416 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-3e1d76d7-3745-40d6-b41f-d7fc09b834b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1578db0219964206875af31847275d90', 'request_id': 'req-3e1d76d7-3745-40d6-b41f-d7fc09b834b2'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:02.061671 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-3e1d76d7-3745-40d6-b41f-d7fc09b834b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1578db0219964206875af31847275d90', 'request_id': 'req-3e1d76d7-3745-40d6-b41f-d7fc09b834b2'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:02.137240 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-3e1d76d7-3745-40d6-b41f-d7fc09b834b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1578db0219964206875af31847275d90', 'request_id': 'req-3e1d76d7-3745-40d6-b41f-d7fc09b834b2'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:02.179242 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-3e1d76d7-3745-40d6-b41f-d7fc09b834b2 admin admin] The token cache is being invalidate because project 1578db0219964206875af31847275d90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:47:02.179242 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-3e1d76d7-3745-40d6-b41f-d7fc09b834b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:02.195548 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-3e1d76d7-3745-40d6-b41f-d7fc09b834b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:02.200261 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1134/2254] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:47:01 2022] DELETE /identity/v3/projects/1578db0219964206875af31847275d90 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:47:02.208434 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e2300-8027-4e23-a601-1fea42956da7 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:02.209813 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e2300-8027-4e23-a601-1fea42956da7 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:02.210431 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e2300-8027-4e23-a601-1fea42956da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:02.253273 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-7b1e2300-8027-4e23-a601-1fea42956da7 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:02.254569 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-7b1e2300-8027-4e23-a601-1fea42956da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:02.409616 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95288842-d523-478b-8293-d021aa7e7c10 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:02.409882 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95288842-d523-478b-8293-d021aa7e7c10 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:02.410583 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95288842-d523-478b-8293-d021aa7e7c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:02.414954 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1121/2255] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:47:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:02.462850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-95288842-d523-478b-8293-d021aa7e7c10 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:02.470249 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-95288842-d523-478b-8293-d021aa7e7c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:02.492320 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1135/2256] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:47:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:02.498389 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e2300-8027-4e23-a601-1fea42956da7 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:02.522300 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e2300-8027-4e23-a601-1fea42956da7 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:02.545658 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9cb8a6-363a-4f72-8b87-650f8b7f5e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:02.554374 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9cb8a6-363a-4f72-8b87-650f8b7f5e67 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:02.554374 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9cb8a6-363a-4f72-8b87-650f8b7f5e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:02.554374 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9cb8a6-363a-4f72-8b87-650f8b7f5e67 admin admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:02.554374 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9cb8a6-363a-4f72-8b87-650f8b7f5e67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:47:02.554374 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9cb8a6-363a-4f72-8b87-650f8b7f5e67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:02.554374 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9cb8a6-363a-4f72-8b87-650f8b7f5e67 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:02.583434 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:02.587897 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1122/2257] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:47:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:47:02.600906 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95288842-d523-478b-8293-d021aa7e7c10 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:02.615964 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95288842-d523-478b-8293-d021aa7e7c10 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:02.649441 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c275ccf2-b8b1-49e8-a1b4-da17a24fa0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:02.650429 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275ccf2-b8b1-49e8-a1b4-da17a24fa0e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:02.650872 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275ccf2-b8b1-49e8-a1b4-da17a24fa0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:02.651290 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275ccf2-b8b1-49e8-a1b4-da17a24fa0e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:02.652458 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c275ccf2-b8b1-49e8-a1b4-da17a24fa0e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:02.655611 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c275ccf2-b8b1-49e8-a1b4-da17a24fa0e4 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:02.678956 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:02.799926 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1136/2258] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:47:02 2022] POST /identity/v3/projects => generated 383 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:47:02.806263 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9cb8a6-363a-4f72-8b87-650f8b7f5e67 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:02.833750 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9cb8a6-363a-4f72-8b87-650f8b7f5e67 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:02.954108 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b3f22-c906-4908-8740-a0df8ed67d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:02.956077 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b3f22-c906-4908-8740-a0df8ed67d01 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:02.956670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b3f22-c906-4908-8740-a0df8ed67d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:02.957837 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b3f22-c906-4908-8740-a0df8ed67d01 admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:02.958748 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404b3f22-c906-4908-8740-a0df8ed67d01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:02.962377 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404b3f22-c906-4908-8740-a0df8ed67d01 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:03.093620 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1123/2259] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:47:02 2022] POST /identity/v3/users => generated 292 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:47:03.104972 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c275ccf2-b8b1-49e8-a1b4-da17a24fa0e4 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:03.119510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c275ccf2-b8b1-49e8-a1b4-da17a24fa0e4 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:03.125621 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf5e102-aac3-4bdd-8e31-3410aa345a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:03.127500 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf5e102-aac3-4bdd-8e31-3410aa345a9d admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:03.130893 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf5e102-aac3-4bdd-8e31-3410aa345a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:03.130893 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf5e102-aac3-4bdd-8e31-3410aa345a9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:03.130893 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf5e102-aac3-4bdd-8e31-3410aa345a9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:03.131184 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf5e102-aac3-4bdd-8e31-3410aa345a9d admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:03.149252 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1137/2260] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:47:03 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:47:03.151192 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b3f22-c906-4908-8740-a0df8ed67d01 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:03.171263 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b3f22-c906-4908-8740-a0df8ed67d01 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:03.178229 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ede3c9-09f9-432f-b642-d8868169cf0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:03.179012 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ede3c9-09f9-432f-b642-d8868169cf0d admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:03.179399 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ede3c9-09f9-432f-b642-d8868169cf0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:03.179831 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ede3c9-09f9-432f-b642-d8868169cf0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:03.180644 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ede3c9-09f9-432f-b642-d8868169cf0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:03.182434 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ede3c9-09f9-432f-b642-d8868169cf0d admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:03.196636 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1124/2261] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:47:03 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:47:03.201443 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf5e102-aac3-4bdd-8e31-3410aa345a9d admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:03.226386 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf5e102-aac3-4bdd-8e31-3410aa345a9d admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:03.234652 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbefe55-7fe3-461c-b639-cb31ed996126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:03.235911 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbefe55-7fe3-461c-b639-cb31ed996126 admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:03.236239 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbefe55-7fe3-461c-b639-cb31ed996126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:03.236778 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbefe55-7fe3-461c-b639-cb31ed996126 admin admin] PATH_INFO: `/v3/projects/78e2dab52c204d42bd859954c6dcdfaf/users/07e40f1e99b14270b20c987d3d580da1/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:03.259980 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbefe55-7fe3-461c-b639-cb31ed996126 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78e2dab52c204d42bd859954c6dcdfaf, user_id=07e40f1e99b14270b20c987d3d580da1, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:03.262896 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbefe55-7fe3-461c-b639-cb31ed996126 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:03.395121 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1138/2262] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:47:03 2022] PUT /identity/v3/projects/78e2dab52c204d42bd859954c6dcdfaf/users/07e40f1e99b14270b20c987d3d580da1/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:03.407007 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b669c-ac77-44cd-afbb-66023210fdfd None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:03.407753 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b669c-ac77-44cd-afbb-66023210fdfd None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:03.408014 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b669c-ac77-44cd-afbb-66023210fdfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:03.456384 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-442b669c-ac77-44cd-afbb-66023210fdfd None None] MFA Rules not processed for user `07e40f1e99b14270b20c987d3d580da1`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:03.458586 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-442b669c-ac77-44cd-afbb-66023210fdfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:03.560543 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1125/2263] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:47:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:47:03.574160 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbefe55-7fe3-461c-b639-cb31ed996126 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:03.577472 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-dfbefe55-7fe3-461c-b639-cb31ed996126 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:03.676872 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbefe55-7fe3-461c-b639-cb31ed996126 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:03.681001 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77686bc4-4a76-4cf9-8db4-31e4c04cd3aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:03.681741 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77686bc4-4a76-4cf9-8db4-31e4c04cd3aa service cinder] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:03.682087 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77686bc4-4a76-4cf9-8db4-31e4c04cd3aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:03.682369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77686bc4-4a76-4cf9-8db4-31e4c04cd3aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:03.701088 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77686bc4-4a76-4cf9-8db4-31e4c04cd3aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:03.707688 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77686bc4-4a76-4cf9-8db4-31e4c04cd3aa service cinder] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:03.722314 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1139/2264] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:47:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:47:08.383537 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442b669c-ac77-44cd-afbb-66023210fdfd None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:08.459832 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442b669c-ac77-44cd-afbb-66023210fdfd None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:08.466150 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab0a2c4-c201-4f6d-9c05-f52269239c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:08.466665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab0a2c4-c201-4f6d-9c05-f52269239c86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:08.466776 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab0a2c4-c201-4f6d-9c05-f52269239c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:08.466998 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab0a2c4-c201-4f6d-9c05-f52269239c86 admin admin] PATH_INFO: `/v3/users/07e40f1e99b14270b20c987d3d580da1` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:08.468391 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab0a2c4-c201-4f6d-9c05-f52269239c86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07e40f1e99b14270b20c987d3d580da1)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:08.470722 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab0a2c4-c201-4f6d-9c05-f52269239c86 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:08.486362 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:08.775423 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-9ab0a2c4-c201-4f6d-9c05-f52269239c86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07e40f1e99b14270b20c987d3d580da1'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:08.783936 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-9ab0a2c4-c201-4f6d-9c05-f52269239c86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07e40f1e99b14270b20c987d3d580da1'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:08.784709 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-9ab0a2c4-c201-4f6d-9c05-f52269239c86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07e40f1e99b14270b20c987d3d580da1'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:08.926075 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-9ab0a2c4-c201-4f6d-9c05-f52269239c86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07e40f1e99b14270b20c987d3d580da1'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:08.944306 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-9ab0a2c4-c201-4f6d-9c05-f52269239c86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07e40f1e99b14270b20c987d3d580da1'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:08.950969 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1126/2265] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:47:08 2022] DELETE /identity/v3/users/07e40f1e99b14270b20c987d3d580da1 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:08.956128 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a962d69-f6ce-4d18-bfd3-ccf62362f64e None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:08.956738 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a962d69-f6ce-4d18-bfd3-ccf62362f64e None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:08.956955 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a962d69-f6ce-4d18-bfd3-ccf62362f64e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:08.979017 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-4a962d69-f6ce-4d18-bfd3-ccf62362f64e None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:08.980273 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-4a962d69-f6ce-4d18-bfd3-ccf62362f64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:09.067857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1140/2266] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:47:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:47:09.080431 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab0a2c4-c201-4f6d-9c05-f52269239c86 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:09.082990 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-9ab0a2c4-c201-4f6d-9c05-f52269239c86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:09.162230 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab0a2c4-c201-4f6d-9c05-f52269239c86 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:09.166528 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd02fa-d4b2-4c7e-a4cd-89a5ca417911 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:09.167434 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd02fa-d4b2-4c7e-a4cd-89a5ca417911 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:09.167718 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd02fa-d4b2-4c7e-a4cd-89a5ca417911 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:09.167995 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd02fa-d4b2-4c7e-a4cd-89a5ca417911 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:09.178688 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd02fa-d4b2-4c7e-a4cd-89a5ca417911 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:09.180194 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd02fa-d4b2-4c7e-a4cd-89a5ca417911 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:09.191814 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1127/2267] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:47:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:47:10.088433 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a962d69-f6ce-4d18-bfd3-ccf62362f64e None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:10.092318 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-4a962d69-f6ce-4d18-bfd3-ccf62362f64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:10.117030 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a962d69-f6ce-4d18-bfd3-ccf62362f64e None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:10.139582 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe9d7ea-25f0-4242-ad03-3f3f1f58bb15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:10.140706 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9d7ea-25f0-4242-ad03-3f3f1f58bb15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:10.145812 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9d7ea-25f0-4242-ad03-3f3f1f58bb15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:10.145812 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9d7ea-25f0-4242-ad03-3f3f1f58bb15 admin admin] PATH_INFO: `/v3/projects/78e2dab52c204d42bd859954c6dcdfaf` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:10.145812 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe9d7ea-25f0-4242-ad03-3f3f1f58bb15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78e2dab52c204d42bd859954c6dcdfaf)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:10.145812 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe9d7ea-25f0-4242-ad03-3f3f1f58bb15 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:10.331935 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-2fe9d7ea-25f0-4242-ad03-3f3f1f58bb15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78e2dab52c204d42bd859954c6dcdfaf', 'request_id': 'req-2fe9d7ea-25f0-4242-ad03-3f3f1f58bb15'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:10.338930 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-2fe9d7ea-25f0-4242-ad03-3f3f1f58bb15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78e2dab52c204d42bd859954c6dcdfaf', 'request_id': 'req-2fe9d7ea-25f0-4242-ad03-3f3f1f58bb15'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:10.348694 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:10.537381 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-2fe9d7ea-25f0-4242-ad03-3f3f1f58bb15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78e2dab52c204d42bd859954c6dcdfaf', 'request_id': 'req-2fe9d7ea-25f0-4242-ad03-3f3f1f58bb15'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:10.551275 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-2fe9d7ea-25f0-4242-ad03-3f3f1f58bb15 admin admin] The token cache is being invalidate because project 78e2dab52c204d42bd859954c6dcdfaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:47:10.551599 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-2fe9d7ea-25f0-4242-ad03-3f3f1f58bb15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:10.552295 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-2fe9d7ea-25f0-4242-ad03-3f3f1f58bb15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:10.553850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1141/2268] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:47:10 2022] DELETE /identity/v3/projects/78e2dab52c204d42bd859954c6dcdfaf => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:47:10.560278 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4936e1-d587-4f39-8ba6-2b43638880c1 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:10.560865 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4936e1-d587-4f39-8ba6-2b43638880c1 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:10.561118 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4936e1-d587-4f39-8ba6-2b43638880c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:10.584961 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-0f4936e1-d587-4f39-8ba6-2b43638880c1 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:10.586096 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-0f4936e1-d587-4f39-8ba6-2b43638880c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:10.666552 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1128/2269] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:47:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:10.676018 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cebacdd-6690-4556-b6da-a3568e5f5e42 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:10.677399 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cebacdd-6690-4556-b6da-a3568e5f5e42 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:10.677399 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cebacdd-6690-4556-b6da-a3568e5f5e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:10.697418 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-5cebacdd-6690-4556-b6da-a3568e5f5e42 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:10.698638 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-5cebacdd-6690-4556-b6da-a3568e5f5e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:10.707581 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1142/2270] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:47:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:10.712390 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4936e1-d587-4f39-8ba6-2b43638880c1 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:10.724069 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4936e1-d587-4f39-8ba6-2b43638880c1 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:10.729082 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97baf646-c350-4468-b29e-f8473c59a927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:10.729731 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97baf646-c350-4468-b29e-f8473c59a927 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:10.729997 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97baf646-c350-4468-b29e-f8473c59a927 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:10.730269 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97baf646-c350-4468-b29e-f8473c59a927 admin admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:10.730930 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97baf646-c350-4468-b29e-f8473c59a927 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:47:10.731252 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97baf646-c350-4468-b29e-f8473c59a927 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:10.733056 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97baf646-c350-4468-b29e-f8473c59a927 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:10.747704 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1129/2271] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:47:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:47:10.752375 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cebacdd-6690-4556-b6da-a3568e5f5e42 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:10.765129 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cebacdd-6690-4556-b6da-a3568e5f5e42 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:10.781926 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f486606-8a19-4cb2-86f6-e90c723000db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:10.782570 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f486606-8a19-4cb2-86f6-e90c723000db admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:10.782862 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f486606-8a19-4cb2-86f6-e90c723000db admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:10.783129 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f486606-8a19-4cb2-86f6-e90c723000db admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:10.783783 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f486606-8a19-4cb2-86f6-e90c723000db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:10.786414 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f486606-8a19-4cb2-86f6-e90c723000db admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:10.901270 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1143/2272] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:47:10 2022] POST /identity/v3/projects => generated 385 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:47:10.905718 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97baf646-c350-4468-b29e-f8473c59a927 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:10.916081 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97baf646-c350-4468-b29e-f8473c59a927 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:10.933486 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:10.984980 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1bdb7-e578-4251-ae01-6c415992975e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:10.985602 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1bdb7-e578-4251-ae01-6c415992975e admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:10.985887 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1bdb7-e578-4251-ae01-6c415992975e admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:10.986164 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1bdb7-e578-4251-ae01-6c415992975e admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:10.986833 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f1bdb7-e578-4251-ae01-6c415992975e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:10.990051 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f1bdb7-e578-4251-ae01-6c415992975e admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:11.137406 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1130/2273] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:47:10 2022] POST /identity/v3/users => generated 293 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:47:11.142973 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f486606-8a19-4cb2-86f6-e90c723000db admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:11.154796 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f486606-8a19-4cb2-86f6-e90c723000db admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:11.162474 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096acb4-9a78-4e61-9980-a5add163ad18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:11.164095 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096acb4-9a78-4e61-9980-a5add163ad18 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:11.164384 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096acb4-9a78-4e61-9980-a5add163ad18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:11.164690 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096acb4-9a78-4e61-9980-a5add163ad18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:11.165358 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4096acb4-9a78-4e61-9980-a5add163ad18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:11.167032 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4096acb4-9a78-4e61-9980-a5add163ad18 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:11.179942 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1144/2274] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:47:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:47:11.189343 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1bdb7-e578-4251-ae01-6c415992975e admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:11.203863 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1bdb7-e578-4251-ae01-6c415992975e admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:11.210072 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b8f63e-2a47-4d51-be45-e773859e427a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:11.212591 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8f63e-2a47-4d51-be45-e773859e427a admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:11.212591 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8f63e-2a47-4d51-be45-e773859e427a admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:11.212591 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8f63e-2a47-4d51-be45-e773859e427a admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:11.212591 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b8f63e-2a47-4d51-be45-e773859e427a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:11.213546 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b8f63e-2a47-4d51-be45-e773859e427a admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:11.230256 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1131/2275] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:47:11 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:47:11.235134 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096acb4-9a78-4e61-9980-a5add163ad18 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:11.246599 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096acb4-9a78-4e61-9980-a5add163ad18 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:11.252985 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae0d0c-6c1a-4412-9c1b-d04e281672f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:11.254176 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae0d0c-6c1a-4412-9c1b-d04e281672f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:11.254176 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae0d0c-6c1a-4412-9c1b-d04e281672f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:11.254176 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae0d0c-6c1a-4412-9c1b-d04e281672f8 admin admin] PATH_INFO: `/v3/projects/de8b104cdda04729a55a5af9d4503c81/users/358428458beb4ad6820c6d40d39390e4/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:11.279231 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ae0d0c-6c1a-4412-9c1b-d04e281672f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de8b104cdda04729a55a5af9d4503c81, user_id=358428458beb4ad6820c6d40d39390e4, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:11.282931 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ae0d0c-6c1a-4412-9c1b-d04e281672f8 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:11.366768 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1145/2276] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:47:11 2022] PUT /identity/v3/projects/de8b104cdda04729a55a5af9d4503c81/users/358428458beb4ad6820c6d40d39390e4/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:11.372456 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef2891-8cba-4957-96c4-bdc36eb769a7 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:11.373020 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef2891-8cba-4957-96c4-bdc36eb769a7 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:11.373172 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef2891-8cba-4957-96c4-bdc36eb769a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:11.396570 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-11ef2891-8cba-4957-96c4-bdc36eb769a7 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:11.398525 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-11ef2891-8cba-4957-96c4-bdc36eb769a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:11.481208 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1132/2277] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:47:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:11.492234 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae0d0c-6c1a-4412-9c1b-d04e281672f8 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:11.493010 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-73ae0d0c-6c1a-4412-9c1b-d04e281672f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:11.571816 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae0d0c-6c1a-4412-9c1b-d04e281672f8 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:11.575710 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca8681e-ffc0-49e6-8272-502c6350df11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:11.576338 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca8681e-ffc0-49e6-8272-502c6350df11 service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:11.576647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca8681e-ffc0-49e6-8272-502c6350df11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:11.576922 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca8681e-ffc0-49e6-8272-502c6350df11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:11.587237 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca8681e-ffc0-49e6-8272-502c6350df11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:11.592599 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca8681e-ffc0-49e6-8272-502c6350df11 service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:11.599928 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1146/2278] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:47:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:47:19.070931 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17832e9-899a-494b-aeea-7a304dc22aba None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:19.071440 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17832e9-899a-494b-aeea-7a304dc22aba None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:19.074346 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17832e9-899a-494b-aeea-7a304dc22aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:19.123345 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-b17832e9-899a-494b-aeea-7a304dc22aba None None] MFA Rules not processed for user `358428458beb4ad6820c6d40d39390e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:19.125382 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-b17832e9-899a-494b-aeea-7a304dc22aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:19.217595 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1133/2279] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:47:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:47:19.223090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca8681e-ffc0-49e6-8272-502c6350df11 service neutron] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:19.235640 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca8681e-ffc0-49e6-8272-502c6350df11 service neutron] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:19.241485 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e56e3c-f1a9-4b9a-866b-744889d9d0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:19.242351 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e56e3c-f1a9-4b9a-866b-744889d9d0a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:19.242935 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e56e3c-f1a9-4b9a-866b-744889d9d0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:19.243216 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e56e3c-f1a9-4b9a-866b-744889d9d0a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:19.243937 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e56e3c-f1a9-4b9a-866b-744889d9d0a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:19.246002 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e56e3c-f1a9-4b9a-866b-744889d9d0a1 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:19.462966 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1147/2280] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:47:19 2022] POST /identity/v3/projects => generated 383 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:47:19.472216 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17832e9-899a-494b-aeea-7a304dc22aba None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:19.485362 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17832e9-899a-494b-aeea-7a304dc22aba None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:19.534522 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:19.620109 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba33e0f-bc00-4d0c-bc46-9e32b9f27cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:19.620793 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba33e0f-bc00-4d0c-bc46-9e32b9f27cb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:19.621078 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba33e0f-bc00-4d0c-bc46-9e32b9f27cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:19.621356 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba33e0f-bc00-4d0c-bc46-9e32b9f27cb7 admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:19.622020 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba33e0f-bc00-4d0c-bc46-9e32b9f27cb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:19.624281 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba33e0f-bc00-4d0c-bc46-9e32b9f27cb7 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:19.766729 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1134/2281] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:47:19 2022] POST /identity/v3/users => generated 298 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:47:19.771242 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e56e3c-f1a9-4b9a-866b-744889d9d0a1 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:19.785901 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e56e3c-f1a9-4b9a-866b-744889d9d0a1 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:19.792896 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4958cda4-61d1-4366-827e-bd90249dc7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:19.793524 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958cda4-61d1-4366-827e-bd90249dc7ff admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:19.793816 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958cda4-61d1-4366-827e-bd90249dc7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:19.794080 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958cda4-61d1-4366-827e-bd90249dc7ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:19.794924 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4958cda4-61d1-4366-827e-bd90249dc7ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:19.796578 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4958cda4-61d1-4366-827e-bd90249dc7ff admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:19.820420 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1148/2282] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:47:19 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:47:19.825113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba33e0f-bc00-4d0c-bc46-9e32b9f27cb7 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:19.849386 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba33e0f-bc00-4d0c-bc46-9e32b9f27cb7 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:19.858731 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec5e08-860f-470f-a8ce-3181380101a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:19.859719 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec5e08-860f-470f-a8ce-3181380101a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:19.860179 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec5e08-860f-470f-a8ce-3181380101a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:19.861718 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec5e08-860f-470f-a8ce-3181380101a7 admin admin] PATH_INFO: `/v3/projects/8d0846e5dc424b1b9e3c96604ca3f51f/users/dcc73f1b94024d4a813ee086de227b38/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:19.897708 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec5e08-860f-470f-a8ce-3181380101a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d0846e5dc424b1b9e3c96604ca3f51f, user_id=dcc73f1b94024d4a813ee086de227b38, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:19.903837 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec5e08-860f-470f-a8ce-3181380101a7 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:20.019806 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1135/2283] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:47:19 2022] PUT /identity/v3/projects/8d0846e5dc424b1b9e3c96604ca3f51f/users/dcc73f1b94024d4a813ee086de227b38/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:20.026486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4958cda4-61d1-4366-827e-bd90249dc7ff admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:20.043484 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4958cda4-61d1-4366-827e-bd90249dc7ff admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:20.116945 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:20.127238 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b992e74-46cc-4a67-bac8-889af964921f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:20.128582 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b992e74-46cc-4a67-bac8-889af964921f admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:20.134652 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b992e74-46cc-4a67-bac8-889af964921f admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:20.134652 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b992e74-46cc-4a67-bac8-889af964921f admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:20.134652 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b992e74-46cc-4a67-bac8-889af964921f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:20.134652 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b992e74-46cc-4a67-bac8-889af964921f admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:20.144327 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1149/2284] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:47:20 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:47:20.148944 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec5e08-860f-470f-a8ce-3181380101a7 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:20.173276 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec5e08-860f-470f-a8ce-3181380101a7 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:20.193706 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d3da7-7633-40f0-bb21-7462c27bbdc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:20.194738 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d3da7-7633-40f0-bb21-7462c27bbdc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:20.195196 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d3da7-7633-40f0-bb21-7462c27bbdc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:20.195625 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d3da7-7633-40f0-bb21-7462c27bbdc7 admin admin] PATH_INFO: `/v3/projects/8d0846e5dc424b1b9e3c96604ca3f51f/users/dcc73f1b94024d4a813ee086de227b38/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:20.198546 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16d3da7-7633-40f0-bb21-7462c27bbdc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d0846e5dc424b1b9e3c96604ca3f51f, user_id=dcc73f1b94024d4a813ee086de227b38, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:20.201728 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16d3da7-7633-40f0-bb21-7462c27bbdc7 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:20.218222 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1136/2285] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:47:20 2022] PUT /identity/v3/projects/8d0846e5dc424b1b9e3c96604ca3f51f/users/dcc73f1b94024d4a813ee086de227b38/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:47:21.475345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aef1ab-7c14-4f65-862b-63de70cf2ed2 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:21.475894 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aef1ab-7c14-4f65-862b-63de70cf2ed2 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:21.475894 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aef1ab-7c14-4f65-862b-63de70cf2ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:21.502039 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-f4aef1ab-7c14-4f65-862b-63de70cf2ed2 None None] MFA Rules not processed for user `926115c2e54741e6a3aca02437309b8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:21.503699 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-f4aef1ab-7c14-4f65-862b-63de70cf2ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:21.579200 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1150/2286] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:47:21 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:28.148190 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a4a7cc-9660-43be-ba4e-4b35e364c9bd None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:28.148862 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a4a7cc-9660-43be-ba4e-4b35e364c9bd None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:28.148862 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a4a7cc-9660-43be-ba4e-4b35e364c9bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:28.195205 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-06a4a7cc-9660-43be-ba4e-4b35e364c9bd None None] MFA Rules not processed for user `dcc73f1b94024d4a813ee086de227b38`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:28.197147 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-06a4a7cc-9660-43be-ba4e-4b35e364c9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:28.308033 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1137/2287] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:47:28 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:28.325630 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aef1ab-7c14-4f65-862b-63de70cf2ed2 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:28.326217 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-f4aef1ab-7c14-4f65-862b-63de70cf2ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:28.400322 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aef1ab-7c14-4f65-862b-63de70cf2ed2 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:28.406316 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b8e556-9e0b-49ad-913e-6d0eb53af76b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9aee9c524a974a248c3bf363444f584a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:28.407367 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8e556-9e0b-49ad-913e-6d0eb53af76b service glance] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:28.407869 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8e556-9e0b-49ad-913e-6d0eb53af76b service glance] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:28.409012 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8e556-9e0b-49ad-913e-6d0eb53af76b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:28.482693 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b8e556-9e0b-49ad-913e-6d0eb53af76b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:28.484148 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b8e556-9e0b-49ad-913e-6d0eb53af76b service glance] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:28.493427 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1151/2288] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:47:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:47:28.910475 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e0340-0cd2-499f-bbfa-7a833f0a49c6 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:28.910991 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e0340-0cd2-499f-bbfa-7a833f0a49c6 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:28.911547 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e0340-0cd2-499f-bbfa-7a833f0a49c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:28.937716 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-be5e0340-0cd2-499f-bbfa-7a833f0a49c6 None None] MFA Rules not processed for user `604380c191c8461bbb937b8d951238bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:28.938940 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-be5e0340-0cd2-499f-bbfa-7a833f0a49c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:29.010302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1138/2289] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Aug 24 05:47:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:29.074441 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b8e556-9e0b-49ad-913e-6d0eb53af76b service glance] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:29.074441 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-07b8e556-9e0b-49ad-913e-6d0eb53af76b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:29.150343 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:29.202594 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b8e556-9e0b-49ad-913e-6d0eb53af76b service glance] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:29.210042 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98b673-2e5d-44cc-985f-8674ffd67aba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb2b6cae4a004539a95fc7dd558db255', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:29.211004 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98b673-2e5d-44cc-985f-8674ffd67aba service swift] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:29.211321 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98b673-2e5d-44cc-985f-8674ffd67aba service swift] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:29.211667 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98b673-2e5d-44cc-985f-8674ffd67aba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:29.222709 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da98b673-2e5d-44cc-985f-8674ffd67aba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:29.224335 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da98b673-2e5d-44cc-985f-8674ffd67aba service swift] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:29.233942 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1152/2290] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Aug 24 05:47:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 05:47:29.498564 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c76e73-4ab9-4d7c-aaf6-193514ee4b7b None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:29.499456 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c76e73-4ab9-4d7c-aaf6-193514ee4b7b None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:29.499601 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c76e73-4ab9-4d7c-aaf6-193514ee4b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:29.567997 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-90c76e73-4ab9-4d7c-aaf6-193514ee4b7b None None] MFA Rules not processed for user `604380c191c8461bbb937b8d951238bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:29.569561 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-90c76e73-4ab9-4d7c-aaf6-193514ee4b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:29.664784 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1139/2291] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Aug 24 05:47:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:29.732659 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98b673-2e5d-44cc-985f-8674ffd67aba service swift] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:29.776055 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98b673-2e5d-44cc-985f-8674ffd67aba service swift] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:29.784835 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e31d4d-0f9b-41b8-b8ad-e97525888089 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb2b6cae4a004539a95fc7dd558db255', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:29.787264 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e31d4d-0f9b-41b8-b8ad-e97525888089 service swift] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:29.787757 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e31d4d-0f9b-41b8-b8ad-e97525888089 service swift] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:29.788203 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e31d4d-0f9b-41b8-b8ad-e97525888089 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:29.806486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e31d4d-0f9b-41b8-b8ad-e97525888089 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:29.811584 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e31d4d-0f9b-41b8-b8ad-e97525888089 service swift] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:29.829444 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1153/2292] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Aug 24 05:47:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 05:47:35.566275 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c76e73-4ab9-4d7c-aaf6-193514ee4b7b None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:35.569312 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-90c76e73-4ab9-4d7c-aaf6-193514ee4b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:35.577085 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:35.700457 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c76e73-4ab9-4d7c-aaf6-193514ee4b7b None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:35.708702 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30ed771-823f-4f17-b086-55020d5133f6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6778b7c5ad19484b877494f143050405', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:35.709439 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ed771-823f-4f17-b086-55020d5133f6 service placement] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:35.709820 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ed771-823f-4f17-b086-55020d5133f6 service placement] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:35.710665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ed771-823f-4f17-b086-55020d5133f6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:35.712957 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-a30ed771-823f-4f17-b086-55020d5133f6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:35.761319 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30ed771-823f-4f17-b086-55020d5133f6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:35.765791 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30ed771-823f-4f17-b086-55020d5133f6 service placement] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:35.793276 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1140/2293] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:47:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:47:39.822945 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=16 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:45.867612 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=31 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:46.129254 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb114e1b-97a0-4def-b8ed-d279f4f8441e None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:46.129254 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb114e1b-97a0-4def-b8ed-d279f4f8441e None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:46.129254 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb114e1b-97a0-4def-b8ed-d279f4f8441e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:46.159332 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-cb114e1b-97a0-4def-b8ed-d279f4f8441e None None] MFA Rules not processed for user `358428458beb4ad6820c6d40d39390e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:46.162580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-cb114e1b-97a0-4def-b8ed-d279f4f8441e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:46.175549 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1154/2294] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:47:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:52.917435 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30ed771-823f-4f17-b086-55020d5133f6 service placement] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:52.919327 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-a30ed771-823f-4f17-b086-55020d5133f6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:53.025825 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30ed771-823f-4f17-b086-55020d5133f6 service placement] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:53.034678 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b101459f-2d45-4d05-b7d1-6e9910cc1e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:53.042770 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101459f-2d45-4d05-b7d1-6e9910cc1e9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:53.043776 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101459f-2d45-4d05-b7d1-6e9910cc1e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:53.044392 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101459f-2d45-4d05-b7d1-6e9910cc1e9b admin admin] PATH_INFO: `/v3/users/9902d63688e24ead8efb75d2fcdbdc14` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:53.046575 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b101459f-2d45-4d05-b7d1-6e9910cc1e9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9902d63688e24ead8efb75d2fcdbdc14)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:53.051511 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b101459f-2d45-4d05-b7d1-6e9910cc1e9b admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:53.378943 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-b101459f-2d45-4d05-b7d1-6e9910cc1e9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9902d63688e24ead8efb75d2fcdbdc14'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:53.393644 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-b101459f-2d45-4d05-b7d1-6e9910cc1e9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9902d63688e24ead8efb75d2fcdbdc14'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:53.395201 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-b101459f-2d45-4d05-b7d1-6e9910cc1e9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9902d63688e24ead8efb75d2fcdbdc14'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:53.491203 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-b101459f-2d45-4d05-b7d1-6e9910cc1e9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9902d63688e24ead8efb75d2fcdbdc14'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:53.527067 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-b101459f-2d45-4d05-b7d1-6e9910cc1e9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9902d63688e24ead8efb75d2fcdbdc14'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:53.540429 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1141/2295] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:47:52 2022] DELETE /identity/v3/users/9902d63688e24ead8efb75d2fcdbdc14 => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:53.540657 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621e9ed-3a4d-4c09-86b3-d83cd6f77bdc None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:53.541510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621e9ed-3a4d-4c09-86b3-d83cd6f77bdc None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:53.541799 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621e9ed-3a4d-4c09-86b3-d83cd6f77bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:53.577795 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-1621e9ed-3a4d-4c09-86b3-d83cd6f77bdc None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:53.579055 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-1621e9ed-3a4d-4c09-86b3-d83cd6f77bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:53.674470 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1155/2296] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:47:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:53.686984 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b101459f-2d45-4d05-b7d1-6e9910cc1e9b admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:53.688683 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-b101459f-2d45-4d05-b7d1-6e9910cc1e9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:53.762506 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b101459f-2d45-4d05-b7d1-6e9910cc1e9b admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:53.766354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fce27f-e0e4-48d0-a5a9-f0fce62238d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:53.767173 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fce27f-e0e4-48d0-a5a9-f0fce62238d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:53.768099 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fce27f-e0e4-48d0-a5a9-f0fce62238d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:53.768382 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fce27f-e0e4-48d0-a5a9-f0fce62238d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:53.779227 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fce27f-e0e4-48d0-a5a9-f0fce62238d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:53.780734 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fce27f-e0e4-48d0-a5a9-f0fce62238d0 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:53.791488 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1142/2297] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:47:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:47:53.885508 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941af45-a288-4279-9e06-9e610f01e655 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:53.885755 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941af45-a288-4279-9e06-9e610f01e655 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:53.886020 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941af45-a288-4279-9e06-9e610f01e655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:53.911056 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-e941af45-a288-4279-9e06-9e610f01e655 None None] MFA Rules not processed for user `604380c191c8461bbb937b8d951238bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:53.912328 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-e941af45-a288-4279-9e06-9e610f01e655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:53.995555 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1156/2298] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Aug 24 05:47:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:47:54.009637 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fce27f-e0e4-48d0-a5a9-f0fce62238d0 service neutron] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:54.012500 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-e0fce27f-e0e4-48d0-a5a9-f0fce62238d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:54.044974 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:54.114763 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fce27f-e0e4-48d0-a5a9-f0fce62238d0 service neutron] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:54.121327 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8502efb4-7a1b-404d-b1e8-8c92a4af3d5f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb2b6cae4a004539a95fc7dd558db255', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:54.122667 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8502efb4-7a1b-404d-b1e8-8c92a4af3d5f service swift] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:54.123218 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8502efb4-7a1b-404d-b1e8-8c92a4af3d5f service swift] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:54.123707 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8502efb4-7a1b-404d-b1e8-8c92a4af3d5f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:54.135245 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8502efb4-7a1b-404d-b1e8-8c92a4af3d5f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:54.137803 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8502efb4-7a1b-404d-b1e8-8c92a4af3d5f service swift] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:54.150197 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1143/2299] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Aug 24 05:47:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 05:47:54.414494 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e941af45-a288-4279-9e06-9e610f01e655 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:54.416061 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-e941af45-a288-4279-9e06-9e610f01e655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:54.445394 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e941af45-a288-4279-9e06-9e610f01e655 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:54.455827 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8bfc3b-ab2f-4a4a-a49a-45afb3a4f311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:54.455827 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8bfc3b-ab2f-4a4a-a49a-45afb3a4f311 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:54.455827 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8bfc3b-ab2f-4a4a-a49a-45afb3a4f311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:54.455827 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8bfc3b-ab2f-4a4a-a49a-45afb3a4f311 admin admin] PATH_INFO: `/v3/projects/efcbca3e4ee149b0bed5ea61c3a42755` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:54.456205 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8bfc3b-ab2f-4a4a-a49a-45afb3a4f311 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efcbca3e4ee149b0bed5ea61c3a42755)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:54.458514 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8bfc3b-ab2f-4a4a-a49a-45afb3a4f311 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:54.666926 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-9b8bfc3b-ab2f-4a4a-a49a-45afb3a4f311 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efcbca3e4ee149b0bed5ea61c3a42755', 'request_id': 'req-9b8bfc3b-ab2f-4a4a-a49a-45afb3a4f311'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:54.676272 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-9b8bfc3b-ab2f-4a4a-a49a-45afb3a4f311 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efcbca3e4ee149b0bed5ea61c3a42755', 'request_id': 'req-9b8bfc3b-ab2f-4a4a-a49a-45afb3a4f311'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:54.747121 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-9b8bfc3b-ab2f-4a4a-a49a-45afb3a4f311 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efcbca3e4ee149b0bed5ea61c3a42755', 'request_id': 'req-9b8bfc3b-ab2f-4a4a-a49a-45afb3a4f311'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:54.769126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-9b8bfc3b-ab2f-4a4a-a49a-45afb3a4f311 admin admin] The token cache is being invalidate because project efcbca3e4ee149b0bed5ea61c3a42755 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:47:54.769684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-9b8bfc3b-ab2f-4a4a-a49a-45afb3a4f311 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:54.770560 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-9b8bfc3b-ab2f-4a4a-a49a-45afb3a4f311 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:54.778502 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea88ae-3b27-4e7c-98c2-621a4b9d5dec None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:54.779038 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea88ae-3b27-4e7c-98c2-621a4b9d5dec None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:54.779473 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea88ae-3b27-4e7c-98c2-621a4b9d5dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:54.783750 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1157/2300] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:47:54 2022] DELETE /identity/v3/projects/efcbca3e4ee149b0bed5ea61c3a42755 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:54.806150 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-b6ea88ae-3b27-4e7c-98c2-621a4b9d5dec None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:54.808299 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-b6ea88ae-3b27-4e7c-98c2-621a4b9d5dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:54.909201 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1144/2301] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:47:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:54.918477 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a175e7-a052-40df-988d-31eb1881a570 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:54.918957 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a175e7-a052-40df-988d-31eb1881a570 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:54.919403 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a175e7-a052-40df-988d-31eb1881a570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:54.948142 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-b7a175e7-a052-40df-988d-31eb1881a570 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:54.949963 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-b7a175e7-a052-40df-988d-31eb1881a570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:54.966427 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea88ae-3b27-4e7c-98c2-621a4b9d5dec None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:54.972555 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1158/2302] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:47:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:54.979476 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea88ae-3b27-4e7c-98c2-621a4b9d5dec None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:54.984711 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7bad84-4721-47b2-b836-46c3531094e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:54.985338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7bad84-4721-47b2-b836-46c3531094e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:54.986778 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7bad84-4721-47b2-b836-46c3531094e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:54.987069 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7bad84-4721-47b2-b836-46c3531094e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:54.987906 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7bad84-4721-47b2-b836-46c3531094e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:47:54.988170 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7bad84-4721-47b2-b836-46c3531094e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:54.990031 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7bad84-4721-47b2-b836-46c3531094e9 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:55.004251 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1145/2303] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:47:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:47:55.009885 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a175e7-a052-40df-988d-31eb1881a570 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:55.022857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a175e7-a052-40df-988d-31eb1881a570 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:55.049625 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716fb7ba-eaec-4683-ba62-f4e3a1bf07d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:55.050355 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716fb7ba-eaec-4683-ba62-f4e3a1bf07d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:55.050710 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716fb7ba-eaec-4683-ba62-f4e3a1bf07d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:55.051061 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716fb7ba-eaec-4683-ba62-f4e3a1bf07d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:55.051769 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716fb7ba-eaec-4683-ba62-f4e3a1bf07d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:55.054132 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716fb7ba-eaec-4683-ba62-f4e3a1bf07d2 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:55.083104 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:55.194584 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1159/2304] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:47:55 2022] POST /identity/v3/projects => generated 375 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:47:55.196478 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7bad84-4721-47b2-b836-46c3531094e9 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:55.215409 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7bad84-4721-47b2-b836-46c3531094e9 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:55.337604 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abf0d97-c148-4b24-83ba-af95fe803f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:55.338314 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf0d97-c148-4b24-83ba-af95fe803f8e admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:55.338598 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf0d97-c148-4b24-83ba-af95fe803f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:55.338874 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf0d97-c148-4b24-83ba-af95fe803f8e admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:55.339555 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abf0d97-c148-4b24-83ba-af95fe803f8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:55.341531 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abf0d97-c148-4b24-83ba-af95fe803f8e admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:55.459707 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1146/2305] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:47:55 2022] POST /identity/v3/users => generated 288 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:47:55.465750 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716fb7ba-eaec-4683-ba62-f4e3a1bf07d2 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:55.493589 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716fb7ba-eaec-4683-ba62-f4e3a1bf07d2 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:55.499604 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adf0ca5-a0a6-4f95-95cf-3ca595a40856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:55.501538 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf0ca5-a0a6-4f95-95cf-3ca595a40856 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:55.502422 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf0ca5-a0a6-4f95-95cf-3ca595a40856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:55.502853 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf0ca5-a0a6-4f95-95cf-3ca595a40856 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:55.503754 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adf0ca5-a0a6-4f95-95cf-3ca595a40856 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:55.505629 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adf0ca5-a0a6-4f95-95cf-3ca595a40856 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:55.545990 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1160/2306] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:47:55 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:47:55.547226 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abf0d97-c148-4b24-83ba-af95fe803f8e admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:55.560554 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abf0d97-c148-4b24-83ba-af95fe803f8e admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:55.567741 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaad37b-5d99-434f-b2fb-0e3c9405ec73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:55.567741 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaad37b-5d99-434f-b2fb-0e3c9405ec73 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:55.567741 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaad37b-5d99-434f-b2fb-0e3c9405ec73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:55.572574 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaad37b-5d99-434f-b2fb-0e3c9405ec73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:55.572574 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaad37b-5d99-434f-b2fb-0e3c9405ec73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:55.572574 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaad37b-5d99-434f-b2fb-0e3c9405ec73 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:55.588462 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1147/2307] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:47:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:47:55.591647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adf0ca5-a0a6-4f95-95cf-3ca595a40856 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:55.607313 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adf0ca5-a0a6-4f95-95cf-3ca595a40856 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:55.607943 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faface6-1bdb-40eb-a806-7bc320783a4d None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:55.608297 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faface6-1bdb-40eb-a806-7bc320783a4d None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:55.608596 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faface6-1bdb-40eb-a806-7bc320783a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:55.617189 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e04fc-23ff-4da6-9f67-2a4dd4c848cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:55.617885 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e04fc-23ff-4da6-9f67-2a4dd4c848cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:55.618210 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e04fc-23ff-4da6-9f67-2a4dd4c848cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:55.618417 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e04fc-23ff-4da6-9f67-2a4dd4c848cd admin admin] PATH_INFO: `/v3/projects/53e1b207a1e9479e88e00371843f7ddd/users/26bdb5e1c6d145e8a8c6792b2534a836/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:55.645643 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-7faface6-1bdb-40eb-a806-7bc320783a4d None None] MFA Rules not processed for user `604380c191c8461bbb937b8d951238bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:55.645643 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-7faface6-1bdb-40eb-a806-7bc320783a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:55.662238 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:55.682785 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5e04fc-23ff-4da6-9f67-2a4dd4c848cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=53e1b207a1e9479e88e00371843f7ddd, user_id=26bdb5e1c6d145e8a8c6792b2534a836, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:55.691660 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5e04fc-23ff-4da6-9f67-2a4dd4c848cd admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:55.732246 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1148/2308] 10.222.0.15 () {58 vars in 967 bytes} [Wed Aug 24 05:47:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:47:55.745669 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faface6-1bdb-40eb-a806-7bc320783a4d None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:55.747133 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-7faface6-1bdb-40eb-a806-7bc320783a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:55.798790 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1161/2309] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:47:55 2022] PUT /identity/v3/projects/53e1b207a1e9479e88e00371843f7ddd/users/26bdb5e1c6d145e8a8c6792b2534a836/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:55.809920 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5725f6-0186-4137-8ec8-f652c70a9f81 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:55.810269 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5725f6-0186-4137-8ec8-f652c70a9f81 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:55.810671 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5725f6-0186-4137-8ec8-f652c70a9f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:55.843786 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-dc5725f6-0186-4137-8ec8-f652c70a9f81 None None] MFA Rules not processed for user `26bdb5e1c6d145e8a8c6792b2534a836`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:55.845501 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-dc5725f6-0186-4137-8ec8-f652c70a9f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:55.856779 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faface6-1bdb-40eb-a806-7bc320783a4d None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:55.950548 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1162/2310] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:47:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:47:55.962860 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5158031f-c825-4e74-9a94-e52fc0f85092 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb2b6cae4a004539a95fc7dd558db255', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:55.963467 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5725f6-0186-4137-8ec8-f652c70a9f81 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:55.963887 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158031f-c825-4e74-9a94-e52fc0f85092 service swift] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:55.964614 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158031f-c825-4e74-9a94-e52fc0f85092 service swift] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:55.964786 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158031f-c825-4e74-9a94-e52fc0f85092 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:55.965323 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-dc5725f6-0186-4137-8ec8-f652c70a9f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:56.058986 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5158031f-c825-4e74-9a94-e52fc0f85092 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:56.061054 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5158031f-c825-4e74-9a94-e52fc0f85092 service swift] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:56.071735 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1149/2311] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Aug 24 05:47:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 05:47:56.091736 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5725f6-0186-4137-8ec8-f652c70a9f81 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:56.098087 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b315d727-3a89-42c2-81dc-ca356d04d407 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:56.101119 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315d727-3a89-42c2-81dc-ca356d04d407 service cinder] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:56.101401 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315d727-3a89-42c2-81dc-ca356d04d407 service cinder] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:56.101693 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315d727-3a89-42c2-81dc-ca356d04d407 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:56.112218 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b315d727-3a89-42c2-81dc-ca356d04d407 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:56.113692 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b315d727-3a89-42c2-81dc-ca356d04d407 service cinder] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:56.130698 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1163/2312] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:47:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:47:59.184161 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a8fa3-1452-4271-af9a-0269c7bb53e3 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:59.184795 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a8fa3-1452-4271-af9a-0269c7bb53e3 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:59.184795 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a8fa3-1452-4271-af9a-0269c7bb53e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:59.204136 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:59.215331 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-a88a8fa3-1452-4271-af9a-0269c7bb53e3 None None] MFA Rules not processed for user `26bdb5e1c6d145e8a8c6792b2534a836`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:59.217486 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-a88a8fa3-1452-4271-af9a-0269c7bb53e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:59.231996 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1150/2313] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:47:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:48:00.740483 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a9a3d-2981-4357-964c-b094657ab340 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:00.741067 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a9a3d-2981-4357-964c-b094657ab340 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:00.741067 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a9a3d-2981-4357-964c-b094657ab340 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:00.742615 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1164/2314] 10.222.0.15 () {54 vars in 921 bytes} [Wed Aug 24 05:48:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:48:00.755012 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bda0ea-b89c-49f9-9ef9-67ae1a905415 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:00.756028 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bda0ea-b89c-49f9-9ef9-67ae1a905415 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:00.756532 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bda0ea-b89c-49f9-9ef9-67ae1a905415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:00.785007 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-64bda0ea-b89c-49f9-9ef9-67ae1a905415 None None] MFA Rules not processed for user `6778b7c5ad19484b877494f143050405`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:00.787138 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-64bda0ea-b89c-49f9-9ef9-67ae1a905415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:00.889226 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1151/2315] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Aug 24 05:48:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:48:00.902635 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80a9a3d-2981-4357-964c-b094657ab340 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:00.907798 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-f80a9a3d-2981-4357-964c-b094657ab340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:00.921570 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bda0ea-b89c-49f9-9ef9-67ae1a905415 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:00.923707 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-64bda0ea-b89c-49f9-9ef9-67ae1a905415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:00.955770 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80a9a3d-2981-4357-964c-b094657ab340 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:00.961744 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40875cbe-c8cc-4746-b632-bb70a4dfd0ba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6778b7c5ad19484b877494f143050405', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:00.962367 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40875cbe-c8cc-4746-b632-bb70a4dfd0ba service placement] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:00.964711 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40875cbe-c8cc-4746-b632-bb70a4dfd0ba service placement] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:00.964711 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40875cbe-c8cc-4746-b632-bb70a4dfd0ba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:00.984455 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40875cbe-c8cc-4746-b632-bb70a4dfd0ba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:00.988642 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40875cbe-c8cc-4746-b632-bb70a4dfd0ba service placement] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:01.006580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1165/2316] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:48:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:48:01.064662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bda0ea-b89c-49f9-9ef9-67ae1a905415 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:01.073000 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17314753-32fe-4199-a2f3-807108a99e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:01.074128 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17314753-32fe-4199-a2f3-807108a99e5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:01.074577 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17314753-32fe-4199-a2f3-807108a99e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:01.075053 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17314753-32fe-4199-a2f3-807108a99e5a admin admin] PATH_INFO: `/v3/users/1c85840c4d104b04b6ed577cce2ca2ff` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:01.077102 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17314753-32fe-4199-a2f3-807108a99e5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c85840c4d104b04b6ed577cce2ca2ff)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:01.081111 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17314753-32fe-4199-a2f3-807108a99e5a admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:01.380646 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-17314753-32fe-4199-a2f3-807108a99e5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c85840c4d104b04b6ed577cce2ca2ff'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:01.388449 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-17314753-32fe-4199-a2f3-807108a99e5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c85840c4d104b04b6ed577cce2ca2ff'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:01.389392 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-17314753-32fe-4199-a2f3-807108a99e5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c85840c4d104b04b6ed577cce2ca2ff'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:01.469604 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-17314753-32fe-4199-a2f3-807108a99e5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c85840c4d104b04b6ed577cce2ca2ff'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:01.498198 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-17314753-32fe-4199-a2f3-807108a99e5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c85840c4d104b04b6ed577cce2ca2ff'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:01.521474 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1152/2317] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:48:00 2022] DELETE /identity/v3/users/1c85840c4d104b04b6ed577cce2ca2ff => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:48:01.871307 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40875cbe-c8cc-4746-b632-bb70a4dfd0ba service placement] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:01.883193 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40875cbe-c8cc-4746-b632-bb70a4dfd0ba service placement] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:01.883527 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17314753-32fe-4199-a2f3-807108a99e5a admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:01.885147 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-17314753-32fe-4199-a2f3-807108a99e5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:01.925874 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:48:01.941081 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:48:01.971718 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6871e0-c03e-47bc-a457-a21fc85c4e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:01.973049 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6871e0-c03e-47bc-a457-a21fc85c4e43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:01.973596 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6871e0-c03e-47bc-a457-a21fc85c4e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:01.974043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6871e0-c03e-47bc-a457-a21fc85c4e43 admin admin] PATH_INFO: `/v3/projects/43149d6590a6468b83faa5f1cd7c6b59` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:01.975699 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6871e0-c03e-47bc-a457-a21fc85c4e43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43149d6590a6468b83faa5f1cd7c6b59)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:01.978271 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6871e0-c03e-47bc-a457-a21fc85c4e43 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:02.013680 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17314753-32fe-4199-a2f3-807108a99e5a admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:02.019299 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f2285-beb4-4743-8118-851812309310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:02.019976 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f2285-beb4-4743-8118-851812309310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:02.020260 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f2285-beb4-4743-8118-851812309310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:02.020569 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f2285-beb4-4743-8118-851812309310 admin admin] PATH_INFO: `/v3/users/27453b6103fc46f28878768474245b24` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:02.021745 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f2285-beb4-4743-8118-851812309310 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27453b6103fc46f28878768474245b24)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:02.023728 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f2285-beb4-4743-8118-851812309310 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:02.136171 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-af6871e0-c03e-47bc-a457-a21fc85c4e43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43149d6590a6468b83faa5f1cd7c6b59', 'request_id': 'req-af6871e0-c03e-47bc-a457-a21fc85c4e43'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:02.144764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-af6871e0-c03e-47bc-a457-a21fc85c4e43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43149d6590a6468b83faa5f1cd7c6b59', 'request_id': 'req-af6871e0-c03e-47bc-a457-a21fc85c4e43'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:02.278939 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-af6871e0-c03e-47bc-a457-a21fc85c4e43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43149d6590a6468b83faa5f1cd7c6b59', 'request_id': 'req-af6871e0-c03e-47bc-a457-a21fc85c4e43'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:02.291827 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-af6871e0-c03e-47bc-a457-a21fc85c4e43 admin admin] The token cache is being invalidate because project 43149d6590a6468b83faa5f1cd7c6b59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:48:02.292225 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-af6871e0-c03e-47bc-a457-a21fc85c4e43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:02.293204 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-af6871e0-c03e-47bc-a457-a21fc85c4e43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:02.294909 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1166/2318] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:48:01 2022] DELETE /identity/v3/projects/43149d6590a6468b83faa5f1cd7c6b59 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:48:02.299163 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-0c2f2285-beb4-4743-8118-851812309310 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27453b6103fc46f28878768474245b24'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:02.310790 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd9a3d-9c82-479c-8376-0e2483b3c947 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:02.310790 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd9a3d-9c82-479c-8376-0e2483b3c947 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:02.310790 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd9a3d-9c82-479c-8376-0e2483b3c947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:02.311136 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-0c2f2285-beb4-4743-8118-851812309310 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27453b6103fc46f28878768474245b24'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:02.311136 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-0c2f2285-beb4-4743-8118-851812309310 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27453b6103fc46f28878768474245b24'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:02.330498 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-c2cd9a3d-9c82-479c-8376-0e2483b3c947 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:02.332029 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-c2cd9a3d-9c82-479c-8376-0e2483b3c947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:02.408434 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-0c2f2285-beb4-4743-8118-851812309310 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27453b6103fc46f28878768474245b24'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:02.431993 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-0c2f2285-beb4-4743-8118-851812309310 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27453b6103fc46f28878768474245b24'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:02.468167 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1153/2319] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:48:01 2022] DELETE /identity/v3/users/27453b6103fc46f28878768474245b24 => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:48:02.478135 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3e8f9-f2f1-4b9c-9d01-9871b4538879 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:02.478708 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3e8f9-f2f1-4b9c-9d01-9871b4538879 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:02.478988 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3e8f9-f2f1-4b9c-9d01-9871b4538879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:02.494416 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1167/2320] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:48:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:48:02.504252 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82948a56-9e8c-4959-a30d-6a7ec49eb0b9 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:02.505196 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82948a56-9e8c-4959-a30d-6a7ec49eb0b9 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:02.516162 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82948a56-9e8c-4959-a30d-6a7ec49eb0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:02.556271 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-c1f3e8f9-f2f1-4b9c-9d01-9871b4538879 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:02.557914 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-c1f3e8f9-f2f1-4b9c-9d01-9871b4538879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:02.558846 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-82948a56-9e8c-4959-a30d-6a7ec49eb0b9 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:02.561566 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-82948a56-9e8c-4959-a30d-6a7ec49eb0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:02.694066 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1154/2321] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:48:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:48:02.703666 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f3e8f9-f2f1-4b9c-9d01-9871b4538879 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:02.709454 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-c1f3e8f9-f2f1-4b9c-9d01-9871b4538879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:02.717625 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1168/2322] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:48:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:48:02.722449 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82948a56-9e8c-4959-a30d-6a7ec49eb0b9 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:02.754112 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82948a56-9e8c-4959-a30d-6a7ec49eb0b9 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:02.776671 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d9db4e-bc7a-4abf-b425-072b63404a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:02.777334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d9db4e-bc7a-4abf-b425-072b63404a04 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:02.777832 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d9db4e-bc7a-4abf-b425-072b63404a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:02.778045 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d9db4e-bc7a-4abf-b425-072b63404a04 admin admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:02.778710 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d9db4e-bc7a-4abf-b425-072b63404a04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:48:02.779025 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d9db4e-bc7a-4abf-b425-072b63404a04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:02.780946 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d9db4e-bc7a-4abf-b425-072b63404a04 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:02.787200 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:48:02.793526 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:48:02.801732 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1169/2323] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:48:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:48:02.807124 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d9db4e-bc7a-4abf-b425-072b63404a04 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:02.821862 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d9db4e-bc7a-4abf-b425-072b63404a04 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:02.828323 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bde1c9-e497-4795-a1ef-7bfb6609a428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:02.829050 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bde1c9-e497-4795-a1ef-7bfb6609a428 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:02.829885 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bde1c9-e497-4795-a1ef-7bfb6609a428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:02.830171 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bde1c9-e497-4795-a1ef-7bfb6609a428 admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:02.830973 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bde1c9-e497-4795-a1ef-7bfb6609a428 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:02.833108 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bde1c9-e497-4795-a1ef-7bfb6609a428 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:02.834513 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f3e8f9-f2f1-4b9c-9d01-9871b4538879 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:02.839472 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9670b30-f4ec-47dc-a71e-1af357bd7e92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:02.840124 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9670b30-f4ec-47dc-a71e-1af357bd7e92 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:02.840746 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9670b30-f4ec-47dc-a71e-1af357bd7e92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:02.841363 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9670b30-f4ec-47dc-a71e-1af357bd7e92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:02.863123 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9670b30-f4ec-47dc-a71e-1af357bd7e92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:02.866193 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9670b30-f4ec-47dc-a71e-1af357bd7e92 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:02.894318 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1155/2324] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:48:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:48:03.064028 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1170/2325] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:48:02 2022] POST /identity/v3/projects => generated 389 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:48:03.069453 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9670b30-f4ec-47dc-a71e-1af357bd7e92 service neutron] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:03.081767 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9670b30-f4ec-47dc-a71e-1af357bd7e92 service neutron] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:03.198171 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57456a92-f191-478d-b59e-af25cc1aaf05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:03.202451 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57456a92-f191-478d-b59e-af25cc1aaf05 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:03.207555 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57456a92-f191-478d-b59e-af25cc1aaf05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:03.207555 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57456a92-f191-478d-b59e-af25cc1aaf05 admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:03.207555 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57456a92-f191-478d-b59e-af25cc1aaf05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:03.207555 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57456a92-f191-478d-b59e-af25cc1aaf05 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:03.346758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1156/2326] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:48:03 2022] POST /identity/v3/users => generated 295 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:48:03.352190 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bde1c9-e497-4795-a1ef-7bfb6609a428 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:03.367145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bde1c9-e497-4795-a1ef-7bfb6609a428 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:03.373078 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e012c4-c910-45ef-b454-ee07254f6bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:03.373726 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e012c4-c910-45ef-b454-ee07254f6bc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:03.374040 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e012c4-c910-45ef-b454-ee07254f6bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:03.375075 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e012c4-c910-45ef-b454-ee07254f6bc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:03.375369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e012c4-c910-45ef-b454-ee07254f6bc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:03.378821 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e012c4-c910-45ef-b454-ee07254f6bc7 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:03.394583 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1171/2327] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:48:03.401356 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57456a92-f191-478d-b59e-af25cc1aaf05 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:03.417074 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57456a92-f191-478d-b59e-af25cc1aaf05 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:03.423994 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9ee398-55d8-4962-a130-d7cb517747dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:03.424735 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ee398-55d8-4962-a130-d7cb517747dc admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:03.425235 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ee398-55d8-4962-a130-d7cb517747dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:03.425725 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ee398-55d8-4962-a130-d7cb517747dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:03.426941 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9ee398-55d8-4962-a130-d7cb517747dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:03.429567 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9ee398-55d8-4962-a130-d7cb517747dc admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:03.448155 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1157/2328] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:48:03.452788 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e012c4-c910-45ef-b454-ee07254f6bc7 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:03.467714 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e012c4-c910-45ef-b454-ee07254f6bc7 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:03.475438 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c2fc6-ad6c-496b-a60d-a742bdef956e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:03.476267 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c2fc6-ad6c-496b-a60d-a742bdef956e admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:03.482967 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c2fc6-ad6c-496b-a60d-a742bdef956e admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:03.482967 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c2fc6-ad6c-496b-a60d-a742bdef956e admin admin] PATH_INFO: `/v3/projects/fbfdbe2181014348a5115ba08c58d4c3/users/c2098b86a92340daa588c7844debde5d/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:03.495136 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427c2fc6-ad6c-496b-a60d-a742bdef956e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbfdbe2181014348a5115ba08c58d4c3, user_id=c2098b86a92340daa588c7844debde5d, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:03.497961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427c2fc6-ad6c-496b-a60d-a742bdef956e admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:03.512114 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9ee398-55d8-4962-a130-d7cb517747dc admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:03.514216 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-6e9ee398-55d8-4962-a130-d7cb517747dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:03.531138 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9ee398-55d8-4962-a130-d7cb517747dc admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:03.549345 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7c9c0b-a3e7-4670-aa12-7ca88c27a596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:03.549893 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c9c0b-a3e7-4670-aa12-7ca88c27a596 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:03.550691 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c9c0b-a3e7-4670-aa12-7ca88c27a596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:03.551029 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c9c0b-a3e7-4670-aa12-7ca88c27a596 admin admin] PATH_INFO: `/v3/projects/376bd21a941b4beea5f3c4e98c6c1468` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:03.553263 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7c9c0b-a3e7-4670-aa12-7ca88c27a596 admin admin] RBAC: Authorizing `identity:delete_project(project_id=376bd21a941b4beea5f3c4e98c6c1468)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:03.556299 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7c9c0b-a3e7-4670-aa12-7ca88c27a596 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:03.624507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1172/2329] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:48:03 2022] PUT /identity/v3/projects/fbfdbe2181014348a5115ba08c58d4c3/users/c2098b86a92340daa588c7844debde5d/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:48:03.637533 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192cc42-76d3-440a-963c-269e368d22ba None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:03.637820 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192cc42-76d3-440a-963c-269e368d22ba None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:03.638214 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192cc42-76d3-440a-963c-269e368d22ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:03.679184 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-6192cc42-76d3-440a-963c-269e368d22ba None None] MFA Rules not processed for user `c2098b86a92340daa588c7844debde5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:03.680621 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-6192cc42-76d3-440a-963c-269e368d22ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:03.743094 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-5d7c9c0b-a3e7-4670-aa12-7ca88c27a596 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '376bd21a941b4beea5f3c4e98c6c1468', 'request_id': 'req-5d7c9c0b-a3e7-4670-aa12-7ca88c27a596'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:03.760233 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-5d7c9c0b-a3e7-4670-aa12-7ca88c27a596 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '376bd21a941b4beea5f3c4e98c6c1468', 'request_id': 'req-5d7c9c0b-a3e7-4670-aa12-7ca88c27a596'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:03.767995 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1173/2330] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:48:03 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:48:03.779101 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192cc42-76d3-440a-963c-269e368d22ba None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:03.780689 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-6192cc42-76d3-440a-963c-269e368d22ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:03.842236 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-5d7c9c0b-a3e7-4670-aa12-7ca88c27a596 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '376bd21a941b4beea5f3c4e98c6c1468', 'request_id': 'req-5d7c9c0b-a3e7-4670-aa12-7ca88c27a596'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:03.857803 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192cc42-76d3-440a-963c-269e368d22ba None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:03.866687 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-5d7c9c0b-a3e7-4670-aa12-7ca88c27a596 admin admin] The token cache is being invalidate because project 376bd21a941b4beea5f3c4e98c6c1468 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:48:03.867129 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-5d7c9c0b-a3e7-4670-aa12-7ca88c27a596 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:03.877497 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-5d7c9c0b-a3e7-4670-aa12-7ca88c27a596 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:03.882342 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1158/2331] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:48:03 2022] DELETE /identity/v3/projects/376bd21a941b4beea5f3c4e98c6c1468 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:48:03.883996 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238cb0be-ff7b-4004-9379-1fe4df97f3e8 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:03.884285 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238cb0be-ff7b-4004-9379-1fe4df97f3e8 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:03.884590 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238cb0be-ff7b-4004-9379-1fe4df97f3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:03.922090 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-238cb0be-ff7b-4004-9379-1fe4df97f3e8 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:03.923427 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-238cb0be-ff7b-4004-9379-1fe4df97f3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:03.935213 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:48:03.950526 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1245b8-8d48-44f5-b73b-95eeee976ce6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:03.951254 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1245b8-8d48-44f5-b73b-95eeee976ce6 service cinder] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:03.951608 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1245b8-8d48-44f5-b73b-95eeee976ce6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:03.952453 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1245b8-8d48-44f5-b73b-95eeee976ce6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:03.956137 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-cd1245b8-8d48-44f5-b73b-95eeee976ce6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:03.962547 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:48:03.991140 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1159/2332] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:48:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:48:03.999262 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023d1d2-81fe-4f4b-8884-96b2d168beea None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:03.999550 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023d1d2-81fe-4f4b-8884-96b2d168beea None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:03.999834 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023d1d2-81fe-4f4b-8884-96b2d168beea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:04.023434 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-7023d1d2-81fe-4f4b-8884-96b2d168beea None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:04.024785 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-7023d1d2-81fe-4f4b-8884-96b2d168beea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:04.034776 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1160/2333] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:48:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:48:04.038903 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7023d1d2-81fe-4f4b-8884-96b2d168beea None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:04.046464 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1245b8-8d48-44f5-b73b-95eeee976ce6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:04.048306 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1245b8-8d48-44f5-b73b-95eeee976ce6 service cinder] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:04.062079 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7023d1d2-81fe-4f4b-8884-96b2d168beea None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:04.062079 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeea9a9-b71b-49ec-b58c-19f192fe3a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:04.062079 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeea9a9-b71b-49ec-b58c-19f192fe3a9c admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:04.062079 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeea9a9-b71b-49ec-b58c-19f192fe3a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:04.062633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeea9a9-b71b-49ec-b58c-19f192fe3a9c admin admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:04.062633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeea9a9-b71b-49ec-b58c-19f192fe3a9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:48:04.062633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeea9a9-b71b-49ec-b58c-19f192fe3a9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:04.064284 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1174/2334] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:48:03 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:48:04.065242 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeea9a9-b71b-49ec-b58c-19f192fe3a9c admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:04.083204 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1161/2335] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:48:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:48:04.088437 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1245b8-8d48-44f5-b73b-95eeee976ce6 service cinder] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:04.101509 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1245b8-8d48-44f5-b73b-95eeee976ce6 service cinder] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:04.110182 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c91e3-0b19-42b5-9469-b0eb5d7fd82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:04.111090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c91e3-0b19-42b5-9469-b0eb5d7fd82c admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:04.111535 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c91e3-0b19-42b5-9469-b0eb5d7fd82c admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:04.111960 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c91e3-0b19-42b5-9469-b0eb5d7fd82c admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:04.112906 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6c91e3-0b19-42b5-9469-b0eb5d7fd82c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:04.115842 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6c91e3-0b19-42b5-9469-b0eb5d7fd82c admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:04.217910 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1175/2336] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:48:04 2022] POST /identity/v3/projects => generated 385 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:48:04.224162 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeea9a9-b71b-49ec-b58c-19f192fe3a9c admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:04.230455 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeea9a9-b71b-49ec-b58c-19f192fe3a9c admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:04.311726 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03262483-054f-4950-a42e-1ceb7a5b4c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:04.312510 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03262483-054f-4950-a42e-1ceb7a5b4c42 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:04.312798 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03262483-054f-4950-a42e-1ceb7a5b4c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:04.313078 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03262483-054f-4950-a42e-1ceb7a5b4c42 admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:04.313730 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03262483-054f-4950-a42e-1ceb7a5b4c42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:04.315918 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03262483-054f-4950-a42e-1ceb7a5b4c42 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:04.420109 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1162/2337] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:48:04 2022] POST /identity/v3/users => generated 293 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:48:04.425531 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c91e3-0b19-42b5-9469-b0eb5d7fd82c admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:04.437623 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c91e3-0b19-42b5-9469-b0eb5d7fd82c admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:04.445235 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a16c5e9-be8d-4df1-b5d7-f56a3ecc7824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:04.445709 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16c5e9-be8d-4df1-b5d7-f56a3ecc7824 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:04.445996 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16c5e9-be8d-4df1-b5d7-f56a3ecc7824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:04.446358 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16c5e9-be8d-4df1-b5d7-f56a3ecc7824 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:04.447978 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a16c5e9-be8d-4df1-b5d7-f56a3ecc7824 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:04.449845 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a16c5e9-be8d-4df1-b5d7-f56a3ecc7824 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:04.465449 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1176/2338] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:48:04.473521 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03262483-054f-4950-a42e-1ceb7a5b4c42 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:04.485025 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03262483-054f-4950-a42e-1ceb7a5b4c42 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:04.490733 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2fbf32-adf3-437c-8518-37553a7a2edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:04.491440 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2fbf32-adf3-437c-8518-37553a7a2edd admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:04.491702 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2fbf32-adf3-437c-8518-37553a7a2edd admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:04.491980 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2fbf32-adf3-437c-8518-37553a7a2edd admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:04.495073 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2fbf32-adf3-437c-8518-37553a7a2edd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:04.495328 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2fbf32-adf3-437c-8518-37553a7a2edd admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:04.522667 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1163/2339] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:48:04.529753 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a16c5e9-be8d-4df1-b5d7-f56a3ecc7824 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:04.541385 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a16c5e9-be8d-4df1-b5d7-f56a3ecc7824 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:04.547743 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3945a880-1105-4b52-bb40-7751c820606e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:04.548780 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945a880-1105-4b52-bb40-7751c820606e admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:04.549064 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945a880-1105-4b52-bb40-7751c820606e admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:04.549451 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945a880-1105-4b52-bb40-7751c820606e admin admin] PATH_INFO: `/v3/projects/45a208f4651b477d982b75123d5d9263/users/d18ff946ec5c4cab92efd913f01fed2b/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:04.569113 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3945a880-1105-4b52-bb40-7751c820606e admin admin] RBAC: Authorizing `identity:create_grant(project_id=45a208f4651b477d982b75123d5d9263, user_id=d18ff946ec5c4cab92efd913f01fed2b, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:04.571882 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3945a880-1105-4b52-bb40-7751c820606e admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:04.709621 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1177/2340] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:48:04 2022] PUT /identity/v3/projects/45a208f4651b477d982b75123d5d9263/users/d18ff946ec5c4cab92efd913f01fed2b/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:48:04.722686 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba04edd7-cc5e-4a6e-9b75-b001f928b143 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:04.722967 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba04edd7-cc5e-4a6e-9b75-b001f928b143 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:04.723243 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba04edd7-cc5e-4a6e-9b75-b001f928b143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:04.754823 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-ba04edd7-cc5e-4a6e-9b75-b001f928b143 None None] MFA Rules not processed for user `d18ff946ec5c4cab92efd913f01fed2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:04.756026 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-ba04edd7-cc5e-4a6e-9b75-b001f928b143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:04.833057 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1164/2341] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:48:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:48:04.846028 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3945a880-1105-4b52-bb40-7751c820606e admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:04.963156 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3945a880-1105-4b52-bb40-7751c820606e admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:04.971980 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65334733-14a4-4b88-bffd-d648ee7ac032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:04.972643 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65334733-14a4-4b88-bffd-d648ee7ac032 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:04.972934 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65334733-14a4-4b88-bffd-d648ee7ac032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:04.973214 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65334733-14a4-4b88-bffd-d648ee7ac032 admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:04.973900 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65334733-14a4-4b88-bffd-d648ee7ac032 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:04.975904 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65334733-14a4-4b88-bffd-d648ee7ac032 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:05.087250 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1178/2342] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:48:04 2022] POST /identity/v3/projects => generated 383 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:48:05.091322 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba04edd7-cc5e-4a6e-9b75-b001f928b143 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:05.103928 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba04edd7-cc5e-4a6e-9b75-b001f928b143 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:05.137383 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:48:05.199418 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248168e4-4e0f-45cb-8fe7-dafe3224e1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:05.200070 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248168e4-4e0f-45cb-8fe7-dafe3224e1ee admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:05.200328 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248168e4-4e0f-45cb-8fe7-dafe3224e1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:05.200634 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248168e4-4e0f-45cb-8fe7-dafe3224e1ee admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:05.201283 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248168e4-4e0f-45cb-8fe7-dafe3224e1ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:05.203209 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248168e4-4e0f-45cb-8fe7-dafe3224e1ee admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:05.359142 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1165/2343] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:48:05 2022] POST /identity/v3/users => generated 292 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:48:05.363799 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65334733-14a4-4b88-bffd-d648ee7ac032 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:05.374514 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65334733-14a4-4b88-bffd-d648ee7ac032 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:05.379671 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb8e04d-7204-4b57-94c4-4e35ecf53e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:05.380322 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8e04d-7204-4b57-94c4-4e35ecf53e81 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:05.380689 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8e04d-7204-4b57-94c4-4e35ecf53e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:05.380964 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8e04d-7204-4b57-94c4-4e35ecf53e81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:05.381627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb8e04d-7204-4b57-94c4-4e35ecf53e81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:05.385899 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb8e04d-7204-4b57-94c4-4e35ecf53e81 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:05.396450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1179/2344] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:48:05.400716 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248168e4-4e0f-45cb-8fe7-dafe3224e1ee admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:05.412043 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248168e4-4e0f-45cb-8fe7-dafe3224e1ee admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:05.416973 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4acc9c-266a-41f4-97f5-a1764046f440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:05.417577 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4acc9c-266a-41f4-97f5-a1764046f440 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:05.417913 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4acc9c-266a-41f4-97f5-a1764046f440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:05.418421 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4acc9c-266a-41f4-97f5-a1764046f440 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:05.419068 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4acc9c-266a-41f4-97f5-a1764046f440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:05.420760 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4acc9c-266a-41f4-97f5-a1764046f440 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:05.435666 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1166/2345] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:48:05.437252 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb8e04d-7204-4b57-94c4-4e35ecf53e81 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:05.449133 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb8e04d-7204-4b57-94c4-4e35ecf53e81 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:05.454667 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bac1de-6410-4e6f-a58c-6dee5c98b71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:05.455649 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bac1de-6410-4e6f-a58c-6dee5c98b71b admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:05.455913 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bac1de-6410-4e6f-a58c-6dee5c98b71b admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:05.456258 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bac1de-6410-4e6f-a58c-6dee5c98b71b admin admin] PATH_INFO: `/v3/projects/4bc9bd9b484a42ac8666ba751bd451d4/users/5c3e7a737f4c492eb7e33673f2f6f0af/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:05.474450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:48:05.477533 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bac1de-6410-4e6f-a58c-6dee5c98b71b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bc9bd9b484a42ac8666ba751bd451d4, user_id=5c3e7a737f4c492eb7e33673f2f6f0af, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:05.480818 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bac1de-6410-4e6f-a58c-6dee5c98b71b admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:05.558825 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1180/2346] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:48:05 2022] PUT /identity/v3/projects/4bc9bd9b484a42ac8666ba751bd451d4/users/5c3e7a737f4c492eb7e33673f2f6f0af/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:48:05.580456 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256f040-9540-4aa6-8649-68f13881060f None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:05.580804 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256f040-9540-4aa6-8649-68f13881060f None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:05.581025 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256f040-9540-4aa6-8649-68f13881060f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:05.622824 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-d256f040-9540-4aa6-8649-68f13881060f None None] MFA Rules not processed for user `5c3e7a737f4c492eb7e33673f2f6f0af`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:05.628704 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-d256f040-9540-4aa6-8649-68f13881060f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:05.726310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1167/2347] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:48:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:48:05.730954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bac1de-6410-4e6f-a58c-6dee5c98b71b admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:05.817947 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bac1de-6410-4e6f-a58c-6dee5c98b71b admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:05.825084 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0302056f-d4ff-46cf-95b0-46a6905d5b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:05.825849 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302056f-d4ff-46cf-95b0-46a6905d5b24 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:05.826229 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302056f-d4ff-46cf-95b0-46a6905d5b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:05.826605 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302056f-d4ff-46cf-95b0-46a6905d5b24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:05.827357 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0302056f-d4ff-46cf-95b0-46a6905d5b24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:05.830470 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0302056f-d4ff-46cf-95b0-46a6905d5b24 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:05.939062 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1181/2348] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:48:05 2022] POST /identity/v3/projects => generated 385 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:48:05.943351 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256f040-9540-4aa6-8649-68f13881060f None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:05.954936 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256f040-9540-4aa6-8649-68f13881060f None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:06.033525 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa84410-ecf0-48b2-bd6d-c734c47fd3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:06.034347 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa84410-ecf0-48b2-bd6d-c734c47fd3cf admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:06.034633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa84410-ecf0-48b2-bd6d-c734c47fd3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:06.034946 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa84410-ecf0-48b2-bd6d-c734c47fd3cf admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:06.035623 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa84410-ecf0-48b2-bd6d-c734c47fd3cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:06.037604 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa84410-ecf0-48b2-bd6d-c734c47fd3cf admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:06.141880 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1168/2349] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:48:05 2022] POST /identity/v3/users => generated 299 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:48:06.145983 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0302056f-d4ff-46cf-95b0-46a6905d5b24 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:06.156976 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0302056f-d4ff-46cf-95b0-46a6905d5b24 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:06.163432 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b37fb6-0a4d-4172-b865-246568bd71e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:06.164085 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b37fb6-0a4d-4172-b865-246568bd71e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:06.164416 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b37fb6-0a4d-4172-b865-246568bd71e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:06.164716 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b37fb6-0a4d-4172-b865-246568bd71e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:06.165370 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b37fb6-0a4d-4172-b865-246568bd71e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:06.167069 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b37fb6-0a4d-4172-b865-246568bd71e3 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:06.180621 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1182/2350] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:48:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:48:06.184927 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa84410-ecf0-48b2-bd6d-c734c47fd3cf admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:06.195589 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa84410-ecf0-48b2-bd6d-c734c47fd3cf admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:06.200431 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed26524-dc7d-4e8b-a82d-8ed4403f4008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:06.201103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed26524-dc7d-4e8b-a82d-8ed4403f4008 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:06.201389 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed26524-dc7d-4e8b-a82d-8ed4403f4008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:06.201667 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed26524-dc7d-4e8b-a82d-8ed4403f4008 admin admin] PATH_INFO: `/v3/projects/8558f99377c448f1a6324cefb21ff4ff/users/1c7fb222a6cc454ca181de888c37710a/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:06.218749 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed26524-dc7d-4e8b-a82d-8ed4403f4008 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8558f99377c448f1a6324cefb21ff4ff, user_id=1c7fb222a6cc454ca181de888c37710a, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:06.221994 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed26524-dc7d-4e8b-a82d-8ed4403f4008 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:06.311105 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1169/2351] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:48:06 2022] PUT /identity/v3/projects/8558f99377c448f1a6324cefb21ff4ff/users/1c7fb222a6cc454ca181de888c37710a/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:48:06.315558 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b37fb6-0a4d-4172-b865-246568bd71e3 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:06.327358 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b37fb6-0a4d-4172-b865-246568bd71e3 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:06.396322 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed0150-cc9c-4f9f-8e04-36b8cca3c104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:06.397404 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed0150-cc9c-4f9f-8e04-36b8cca3c104 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:06.397908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed0150-cc9c-4f9f-8e04-36b8cca3c104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:06.398416 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed0150-cc9c-4f9f-8e04-36b8cca3c104 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:06.399617 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ed0150-cc9c-4f9f-8e04-36b8cca3c104 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:06.403366 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ed0150-cc9c-4f9f-8e04-36b8cca3c104 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:06.417263 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1183/2352] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:48:06 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:48:06.422678 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed26524-dc7d-4e8b-a82d-8ed4403f4008 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:06.428726 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:48:06.434095 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed26524-dc7d-4e8b-a82d-8ed4403f4008 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:06.439297 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8456f363-a272-44e2-bafc-e4fbd99a7c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:06.444423 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8456f363-a272-44e2-bafc-e4fbd99a7c48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:06.444423 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8456f363-a272-44e2-bafc-e4fbd99a7c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:06.444423 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8456f363-a272-44e2-bafc-e4fbd99a7c48 admin admin] PATH_INFO: `/v3/projects/8558f99377c448f1a6324cefb21ff4ff/users/1c7fb222a6cc454ca181de888c37710a/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:06.444423 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8456f363-a272-44e2-bafc-e4fbd99a7c48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8558f99377c448f1a6324cefb21ff4ff, user_id=1c7fb222a6cc454ca181de888c37710a, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:06.445228 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8456f363-a272-44e2-bafc-e4fbd99a7c48 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:06.457557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1170/2353] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:48:06 2022] PUT /identity/v3/projects/8558f99377c448f1a6324cefb21ff4ff/users/1c7fb222a6cc454ca181de888c37710a/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:48:06.468122 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29aed8-18d8-4bf9-9a4a-9190857fb1e8 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:06.468460 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29aed8-18d8-4bf9-9a4a-9190857fb1e8 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:06.468741 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29aed8-18d8-4bf9-9a4a-9190857fb1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:06.504467 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-ac29aed8-18d8-4bf9-9a4a-9190857fb1e8 None None] MFA Rules not processed for user `1c7fb222a6cc454ca181de888c37710a`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:06.506611 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-ac29aed8-18d8-4bf9-9a4a-9190857fb1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:06.570785 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:48:06.592259 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1184/2354] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:48:06 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:48:06.605673 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8456f363-a272-44e2-bafc-e4fbd99a7c48 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:06.607642 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-8456f363-a272-44e2-bafc-e4fbd99a7c48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:06.671733 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8456f363-a272-44e2-bafc-e4fbd99a7c48 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:06.675696 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b44992-dd92-4af1-b5ca-959a6a37efea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:06.676313 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b44992-dd92-4af1-b5ca-959a6a37efea service cinder] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:06.676723 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b44992-dd92-4af1-b5ca-959a6a37efea service cinder] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:06.677012 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b44992-dd92-4af1-b5ca-959a6a37efea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:06.749123 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b44992-dd92-4af1-b5ca-959a6a37efea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:06.750621 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b44992-dd92-4af1-b5ca-959a6a37efea service cinder] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:06.761313 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1171/2355] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:48:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:48:16.365380 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac29aed8-18d8-4bf9-9a4a-9190857fb1e8 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:16.375557 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac29aed8-18d8-4bf9-9a4a-9190857fb1e8 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:16.379521 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c80e5-35ad-4b38-ba73-aed3f1a13980 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:16.380495 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c80e5-35ad-4b38-ba73-aed3f1a13980 service cinder] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:16.380495 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c80e5-35ad-4b38-ba73-aed3f1a13980 service cinder] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:16.380495 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c80e5-35ad-4b38-ba73-aed3f1a13980 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:16.458048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9c80e5-35ad-4b38-ba73-aed3f1a13980 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:16.460328 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9c80e5-35ad-4b38-ba73-aed3f1a13980 service cinder] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:16.471258 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1185/2356] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:48:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:48:16.756592 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=52,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:48:19.994438 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b44992-dd92-4af1-b5ca-959a6a37efea service cinder] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:20.003339 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b44992-dd92-4af1-b5ca-959a6a37efea service cinder] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:20.007043 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c792c413-c3f2-4e53-9a0c-112c91deb4c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:20.008003 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c792c413-c3f2-4e53-9a0c-112c91deb4c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:20.008288 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c792c413-c3f2-4e53-9a0c-112c91deb4c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:20.008600 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c792c413-c3f2-4e53-9a0c-112c91deb4c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:20.019268 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c792c413-c3f2-4e53-9a0c-112c91deb4c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:20.020753 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c792c413-c3f2-4e53-9a0c-112c91deb4c5 service cinder] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:20.031283 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1172/2357] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:48:19 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:48:26.465722 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=34 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:48:30.026605 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=6 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:48:32.416641 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c80e5-35ad-4b38-ba73-aed3f1a13980 service cinder] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:32.497157 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c80e5-35ad-4b38-ba73-aed3f1a13980 service cinder] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:32.503769 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3477ea-a9b0-44c7-aca5-7f971b521587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:32.504504 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3477ea-a9b0-44c7-aca5-7f971b521587 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:32.504798 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3477ea-a9b0-44c7-aca5-7f971b521587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:32.505076 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3477ea-a9b0-44c7-aca5-7f971b521587 admin admin] PATH_INFO: `/v3/users/d18ff946ec5c4cab92efd913f01fed2b` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:32.506831 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3477ea-a9b0-44c7-aca5-7f971b521587 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d18ff946ec5c4cab92efd913f01fed2b)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:32.508286 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3477ea-a9b0-44c7-aca5-7f971b521587 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:32.738831 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-0b3477ea-a9b0-44c7-aca5-7f971b521587 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd18ff946ec5c4cab92efd913f01fed2b'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:32.804427 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-0b3477ea-a9b0-44c7-aca5-7f971b521587 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd18ff946ec5c4cab92efd913f01fed2b'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:32.813538 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-0b3477ea-a9b0-44c7-aca5-7f971b521587 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd18ff946ec5c4cab92efd913f01fed2b'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:32.833713 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-0b3477ea-a9b0-44c7-aca5-7f971b521587 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd18ff946ec5c4cab92efd913f01fed2b'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:32.837207 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-0b3477ea-a9b0-44c7-aca5-7f971b521587 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd18ff946ec5c4cab92efd913f01fed2b'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:32.843710 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1186/2358] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:48:32 2022] DELETE /identity/v3/users/d18ff946ec5c4cab92efd913f01fed2b => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:48:32.848674 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a51c06-60b9-499c-8a6c-39ee444d429d None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:32.849245 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a51c06-60b9-499c-8a6c-39ee444d429d None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:32.849704 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a51c06-60b9-499c-8a6c-39ee444d429d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:32.875054 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-06a51c06-60b9-499c-8a6c-39ee444d429d None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:32.877270 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-06a51c06-60b9-499c-8a6c-39ee444d429d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:33.003395 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1173/2359] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:48:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:48:33.010644 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3477ea-a9b0-44c7-aca5-7f971b521587 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:33.012824 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-0b3477ea-a9b0-44c7-aca5-7f971b521587 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:33.120568 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3477ea-a9b0-44c7-aca5-7f971b521587 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:33.127129 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef1fbf-3a02-41f2-81bb-98ff9a4b40f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:33.128296 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef1fbf-3a02-41f2-81bb-98ff9a4b40f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:33.128614 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef1fbf-3a02-41f2-81bb-98ff9a4b40f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:33.128887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef1fbf-3a02-41f2-81bb-98ff9a4b40f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:33.139633 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfef1fbf-3a02-41f2-81bb-98ff9a4b40f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:33.141756 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfef1fbf-3a02-41f2-81bb-98ff9a4b40f3 service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:33.160023 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1187/2360] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:48:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:48:33.958923 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a51c06-60b9-499c-8a6c-39ee444d429d None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:33.961932 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-06a51c06-60b9-499c-8a6c-39ee444d429d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:33.974427 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a51c06-60b9-499c-8a6c-39ee444d429d None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:33.979328 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797c5b37-814b-4541-8bab-7ba0a88ecb19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:33.980035 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c5b37-814b-4541-8bab-7ba0a88ecb19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:33.980547 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c5b37-814b-4541-8bab-7ba0a88ecb19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:33.981051 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c5b37-814b-4541-8bab-7ba0a88ecb19 admin admin] PATH_INFO: `/v3/projects/45a208f4651b477d982b75123d5d9263` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:33.983876 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797c5b37-814b-4541-8bab-7ba0a88ecb19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45a208f4651b477d982b75123d5d9263)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:33.986220 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797c5b37-814b-4541-8bab-7ba0a88ecb19 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:34.149938 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-797c5b37-814b-4541-8bab-7ba0a88ecb19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45a208f4651b477d982b75123d5d9263', 'request_id': 'req-797c5b37-814b-4541-8bab-7ba0a88ecb19'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:34.156205 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-797c5b37-814b-4541-8bab-7ba0a88ecb19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45a208f4651b477d982b75123d5d9263', 'request_id': 'req-797c5b37-814b-4541-8bab-7ba0a88ecb19'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:34.341178 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-797c5b37-814b-4541-8bab-7ba0a88ecb19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45a208f4651b477d982b75123d5d9263', 'request_id': 'req-797c5b37-814b-4541-8bab-7ba0a88ecb19'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:34.353572 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-797c5b37-814b-4541-8bab-7ba0a88ecb19 admin admin] The token cache is being invalidate because project 45a208f4651b477d982b75123d5d9263 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:48:34.353869 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-797c5b37-814b-4541-8bab-7ba0a88ecb19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:34.356176 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-797c5b37-814b-4541-8bab-7ba0a88ecb19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:34.359336 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1174/2361] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:48:33 2022] DELETE /identity/v3/projects/45a208f4651b477d982b75123d5d9263 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:48:34.360426 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef1fbf-3a02-41f2-81bb-98ff9a4b40f3 service neutron] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:34.371577 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef1fbf-3a02-41f2-81bb-98ff9a4b40f3 service neutron] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:34.392230 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:48:34.455296 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de52e4-b3bb-41dd-b8a9-d6d0aa04794d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:34.456149 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de52e4-b3bb-41dd-b8a9-d6d0aa04794d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:34.456601 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de52e4-b3bb-41dd-b8a9-d6d0aa04794d admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:34.456975 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de52e4-b3bb-41dd-b8a9-d6d0aa04794d admin admin] PATH_INFO: `/v3/users/5c3e7a737f4c492eb7e33673f2f6f0af` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:34.459193 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65de52e4-b3bb-41dd-b8a9-d6d0aa04794d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c3e7a737f4c492eb7e33673f2f6f0af)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:34.463375 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65de52e4-b3bb-41dd-b8a9-d6d0aa04794d admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:34.719491 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-65de52e4-b3bb-41dd-b8a9-d6d0aa04794d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c3e7a737f4c492eb7e33673f2f6f0af'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:34.832712 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-65de52e4-b3bb-41dd-b8a9-d6d0aa04794d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c3e7a737f4c492eb7e33673f2f6f0af'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:34.840165 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-65de52e4-b3bb-41dd-b8a9-d6d0aa04794d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c3e7a737f4c492eb7e33673f2f6f0af'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:34.852638 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-65de52e4-b3bb-41dd-b8a9-d6d0aa04794d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c3e7a737f4c492eb7e33673f2f6f0af'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:34.853343 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-65de52e4-b3bb-41dd-b8a9-d6d0aa04794d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c3e7a737f4c492eb7e33673f2f6f0af'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:34.859392 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1188/2362] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:48:34 2022] DELETE /identity/v3/users/5c3e7a737f4c492eb7e33673f2f6f0af => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:48:34.935337 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797c5b37-814b-4541-8bab-7ba0a88ecb19 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:34.943837 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-797c5b37-814b-4541-8bab-7ba0a88ecb19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:34.950428 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797c5b37-814b-4541-8bab-7ba0a88ecb19 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:35.014948 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a112b-9438-4a63-ba69-74e37e164200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:35.016084 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a112b-9438-4a63-ba69-74e37e164200 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:35.016653 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a112b-9438-4a63-ba69-74e37e164200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:35.017661 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a112b-9438-4a63-ba69-74e37e164200 admin admin] PATH_INFO: `/v3/users/c2098b86a92340daa588c7844debde5d` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:35.019909 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6a112b-9438-4a63-ba69-74e37e164200 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2098b86a92340daa588c7844debde5d)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:35.023582 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6a112b-9438-4a63-ba69-74e37e164200 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:35.139520 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:48:35.271262 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-9f6a112b-9438-4a63-ba69-74e37e164200 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2098b86a92340daa588c7844debde5d'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:35.278568 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-9f6a112b-9438-4a63-ba69-74e37e164200 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2098b86a92340daa588c7844debde5d'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:35.279399 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-9f6a112b-9438-4a63-ba69-74e37e164200 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2098b86a92340daa588c7844debde5d'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:35.387856 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de52e4-b3bb-41dd-b8a9-d6d0aa04794d admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:35.388827 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-65de52e4-b3bb-41dd-b8a9-d6d0aa04794d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:35.394522 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-9f6a112b-9438-4a63-ba69-74e37e164200 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2098b86a92340daa588c7844debde5d'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:35.407932 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-9f6a112b-9438-4a63-ba69-74e37e164200 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2098b86a92340daa588c7844debde5d'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:35.409750 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de52e4-b3bb-41dd-b8a9-d6d0aa04794d admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:35.414571 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89754da3-9ca2-4565-8dc8-7e60fdf3bd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:35.415313 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1175/2363] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:48:34 2022] DELETE /identity/v3/users/c2098b86a92340daa588c7844debde5d => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:48:35.415498 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89754da3-9ca2-4565-8dc8-7e60fdf3bd6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:35.415498 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89754da3-9ca2-4565-8dc8-7e60fdf3bd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:35.415947 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89754da3-9ca2-4565-8dc8-7e60fdf3bd6a admin admin] PATH_INFO: `/v3/projects/4bc9bd9b484a42ac8666ba751bd451d4` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:35.417180 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89754da3-9ca2-4565-8dc8-7e60fdf3bd6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bc9bd9b484a42ac8666ba751bd451d4)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:35.419602 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89754da3-9ca2-4565-8dc8-7e60fdf3bd6a admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:35.420034 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cafea9-797a-4846-aafc-427776681863 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:35.420616 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cafea9-797a-4846-aafc-427776681863 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:35.421270 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cafea9-797a-4846-aafc-427776681863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:35.441477 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-86cafea9-797a-4846-aafc-427776681863 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:35.442786 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-86cafea9-797a-4846-aafc-427776681863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:35.505065 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1176/2364] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:48:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:48:35.514884 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cafea9-797a-4846-aafc-427776681863 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:35.516289 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-86cafea9-797a-4846-aafc-427776681863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:35.547748 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-89754da3-9ca2-4565-8dc8-7e60fdf3bd6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bc9bd9b484a42ac8666ba751bd451d4', 'request_id': 'req-89754da3-9ca2-4565-8dc8-7e60fdf3bd6a'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:35.554445 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-89754da3-9ca2-4565-8dc8-7e60fdf3bd6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bc9bd9b484a42ac8666ba751bd451d4', 'request_id': 'req-89754da3-9ca2-4565-8dc8-7e60fdf3bd6a'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:35.578337 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cafea9-797a-4846-aafc-427776681863 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:35.632295 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8d7e0-a272-4ddb-a183-799dfc770bb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:35.632975 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8d7e0-a272-4ddb-a183-799dfc770bb2 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:35.633256 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8d7e0-a272-4ddb-a183-799dfc770bb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:35.633533 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8d7e0-a272-4ddb-a183-799dfc770bb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:35.647189 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:48:35.675681 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-89754da3-9ca2-4565-8dc8-7e60fdf3bd6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bc9bd9b484a42ac8666ba751bd451d4', 'request_id': 'req-89754da3-9ca2-4565-8dc8-7e60fdf3bd6a'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:35.687880 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-89754da3-9ca2-4565-8dc8-7e60fdf3bd6a admin admin] The token cache is being invalidate because project 4bc9bd9b484a42ac8666ba751bd451d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:48:35.688240 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-89754da3-9ca2-4565-8dc8-7e60fdf3bd6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:35.689509 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-89754da3-9ca2-4565-8dc8-7e60fdf3bd6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:35.691277 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1189/2365] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:48:35 2022] DELETE /identity/v3/projects/4bc9bd9b484a42ac8666ba751bd451d4 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:48:35.695359 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89754da3-9ca2-4565-8dc8-7e60fdf3bd6a admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:35.696753 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-89754da3-9ca2-4565-8dc8-7e60fdf3bd6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:35.700926 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d8d7e0-a272-4ddb-a183-799dfc770bb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:35.702528 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d8d7e0-a272-4ddb-a183-799dfc770bb2 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:35.711710 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89754da3-9ca2-4565-8dc8-7e60fdf3bd6a admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:35.715141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1177/2366] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:48:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:48:35.717377 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea8d5e-280b-4f5a-9ec4-bc0b7eb33207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:35.718221 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea8d5e-280b-4f5a-9ec4-bc0b7eb33207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:35.724013 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea8d5e-280b-4f5a-9ec4-bc0b7eb33207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:35.724013 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea8d5e-280b-4f5a-9ec4-bc0b7eb33207 admin admin] PATH_INFO: `/v3/users/1c7fb222a6cc454ca181de888c37710a` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:35.724013 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ea8d5e-280b-4f5a-9ec4-bc0b7eb33207 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c7fb222a6cc454ca181de888c37710a)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:35.724013 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ea8d5e-280b-4f5a-9ec4-bc0b7eb33207 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:35.760928 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:48:35.981373 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-93ea8d5e-280b-4f5a-9ec4-bc0b7eb33207 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c7fb222a6cc454ca181de888c37710a'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:36.085272 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-93ea8d5e-280b-4f5a-9ec4-bc0b7eb33207 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c7fb222a6cc454ca181de888c37710a'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:36.093449 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-93ea8d5e-280b-4f5a-9ec4-bc0b7eb33207 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c7fb222a6cc454ca181de888c37710a'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:36.107739 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-93ea8d5e-280b-4f5a-9ec4-bc0b7eb33207 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c7fb222a6cc454ca181de888c37710a'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:36.108514 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-93ea8d5e-280b-4f5a-9ec4-bc0b7eb33207 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c7fb222a6cc454ca181de888c37710a'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:36.115917 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1190/2367] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:48:35 2022] DELETE /identity/v3/users/1c7fb222a6cc454ca181de888c37710a => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:48:36.234427 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8d7e0-a272-4ddb-a183-799dfc770bb2 service neutron] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:36.237649 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-a6d8d7e0-a272-4ddb-a183-799dfc770bb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:36.273812 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8d7e0-a272-4ddb-a183-799dfc770bb2 service neutron] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:36.350969 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae4e5f-a9b1-4a55-ad6c-c8a54f48f3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:36.351602 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae4e5f-a9b1-4a55-ad6c-c8a54f48f3c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:36.351880 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae4e5f-a9b1-4a55-ad6c-c8a54f48f3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:36.352149 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae4e5f-a9b1-4a55-ad6c-c8a54f48f3c8 admin admin] PATH_INFO: `/v3/projects/fbfdbe2181014348a5115ba08c58d4c3` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:36.353818 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deae4e5f-a9b1-4a55-ad6c-c8a54f48f3c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbfdbe2181014348a5115ba08c58d4c3)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:36.355269 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deae4e5f-a9b1-4a55-ad6c-c8a54f48f3c8 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:36.502908 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-deae4e5f-a9b1-4a55-ad6c-c8a54f48f3c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbfdbe2181014348a5115ba08c58d4c3', 'request_id': 'req-deae4e5f-a9b1-4a55-ad6c-c8a54f48f3c8'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:36.515115 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-deae4e5f-a9b1-4a55-ad6c-c8a54f48f3c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbfdbe2181014348a5115ba08c58d4c3', 'request_id': 'req-deae4e5f-a9b1-4a55-ad6c-c8a54f48f3c8'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:36.604451 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-deae4e5f-a9b1-4a55-ad6c-c8a54f48f3c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbfdbe2181014348a5115ba08c58d4c3', 'request_id': 'req-deae4e5f-a9b1-4a55-ad6c-c8a54f48f3c8'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:36.615995 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-deae4e5f-a9b1-4a55-ad6c-c8a54f48f3c8 admin admin] The token cache is being invalidate because project fbfdbe2181014348a5115ba08c58d4c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:48:36.616284 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-deae4e5f-a9b1-4a55-ad6c-c8a54f48f3c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:36.616988 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-deae4e5f-a9b1-4a55-ad6c-c8a54f48f3c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:36.618513 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1178/2368] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:48:36 2022] DELETE /identity/v3/projects/fbfdbe2181014348a5115ba08c58d4c3 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:48:36.625131 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c674da-34df-42ef-91b7-652e340b634e None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:36.625615 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c674da-34df-42ef-91b7-652e340b634e None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:36.625739 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c674da-34df-42ef-91b7-652e340b634e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:36.645181 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae4e5f-a9b1-4a55-ad6c-c8a54f48f3c8 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:36.646607 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-deae4e5f-a9b1-4a55-ad6c-c8a54f48f3c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:36.658792 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-03c674da-34df-42ef-91b7-652e340b634e None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:36.660259 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-03c674da-34df-42ef-91b7-652e340b634e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:36.704009 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:48:36.729523 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae4e5f-a9b1-4a55-ad6c-c8a54f48f3c8 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:36.730721 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1191/2369] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:48:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:48:36.734314 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91ba33-a8db-4123-b62a-ecae8095f903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:36.734947 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91ba33-a8db-4123-b62a-ecae8095f903 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:36.735227 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91ba33-a8db-4123-b62a-ecae8095f903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:36.735534 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91ba33-a8db-4123-b62a-ecae8095f903 admin admin] PATH_INFO: `/v3/projects/8558f99377c448f1a6324cefb21ff4ff` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:36.737270 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb91ba33-a8db-4123-b62a-ecae8095f903 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8558f99377c448f1a6324cefb21ff4ff)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:36.738743 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dbf120-d859-491a-92b4-1c5bf42c3781 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:36.739037 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dbf120-d859-491a-92b4-1c5bf42c3781 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:36.739237 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb91ba33-a8db-4123-b62a-ecae8095f903 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:36.739347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dbf120-d859-491a-92b4-1c5bf42c3781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:36.759192 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-a0dbf120-d859-491a-92b4-1c5bf42c3781 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:36.760447 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-a0dbf120-d859-491a-92b4-1c5bf42c3781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:36.769676 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1192/2370] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:48:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:48:36.774343 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dbf120-d859-491a-92b4-1c5bf42c3781 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:36.783828 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dbf120-d859-491a-92b4-1c5bf42c3781 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:36.788560 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e5e50-de6d-4f24-9350-3074c40824b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:36.789168 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e5e50-de6d-4f24-9350-3074c40824b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:36.789434 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e5e50-de6d-4f24-9350-3074c40824b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:36.789711 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e5e50-de6d-4f24-9350-3074c40824b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:36.790656 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19e5e50-de6d-4f24-9350-3074c40824b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:48:36.791016 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19e5e50-de6d-4f24-9350-3074c40824b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:36.792937 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19e5e50-de6d-4f24-9350-3074c40824b8 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:36.805109 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1193/2371] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:48:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:48:36.809413 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e5e50-de6d-4f24-9350-3074c40824b8 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:36.820296 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e5e50-de6d-4f24-9350-3074c40824b8 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:36.825332 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc0cbf9-8d53-44b7-a826-7d161aaf7703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:36.825943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0cbf9-8d53-44b7-a826-7d161aaf7703 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:36.826228 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0cbf9-8d53-44b7-a826-7d161aaf7703 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:36.826506 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0cbf9-8d53-44b7-a826-7d161aaf7703 admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:36.827149 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc0cbf9-8d53-44b7-a826-7d161aaf7703 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:36.829118 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc0cbf9-8d53-44b7-a826-7d161aaf7703 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:36.884025 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-fb91ba33-a8db-4123-b62a-ecae8095f903 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8558f99377c448f1a6324cefb21ff4ff', 'request_id': 'req-fb91ba33-a8db-4123-b62a-ecae8095f903'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:36.890126 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-fb91ba33-a8db-4123-b62a-ecae8095f903 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8558f99377c448f1a6324cefb21ff4ff', 'request_id': 'req-fb91ba33-a8db-4123-b62a-ecae8095f903'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:36.977598 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1194/2372] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:48:36 2022] POST /identity/v3/projects => generated 379 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:48:36.982328 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-fb91ba33-a8db-4123-b62a-ecae8095f903 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8558f99377c448f1a6324cefb21ff4ff', 'request_id': 'req-fb91ba33-a8db-4123-b62a-ecae8095f903'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:36.982557 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc0cbf9-8d53-44b7-a826-7d161aaf7703 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:36.999682 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-fb91ba33-a8db-4123-b62a-ecae8095f903 admin admin] The token cache is being invalidate because project 8558f99377c448f1a6324cefb21ff4ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:48:37.000024 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-fb91ba33-a8db-4123-b62a-ecae8095f903 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:37.000674 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-fb91ba33-a8db-4123-b62a-ecae8095f903 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:37.003392 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1179/2373] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:48:36 2022] DELETE /identity/v3/projects/8558f99377c448f1a6324cefb21ff4ff => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:48:37.007843 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ba91f-fb54-45e2-8acb-bcc4c1b221a0 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:37.008130 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ba91f-fb54-45e2-8acb-bcc4c1b221a0 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:37.008433 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ba91f-fb54-45e2-8acb-bcc4c1b221a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:37.031100 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-228ba91f-fb54-45e2-8acb-bcc4c1b221a0 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:37.033500 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-228ba91f-fb54-45e2-8acb-bcc4c1b221a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:37.068139 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:48:37.076592 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc0cbf9-8d53-44b7-a826-7d161aaf7703 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:37.083497 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb467e-7226-4eea-965c-fb07a297c2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:37.085070 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb467e-7226-4eea-965c-fb07a297c2da admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:37.087718 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb467e-7226-4eea-965c-fb07a297c2da admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:37.087718 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb467e-7226-4eea-965c-fb07a297c2da admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:37.087718 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdb467e-7226-4eea-965c-fb07a297c2da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:37.088516 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdb467e-7226-4eea-965c-fb07a297c2da admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:37.109933 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1180/2374] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:48:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:48:37.119781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a67651-449c-4e7e-9190-06a89eabf23d None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:37.135763 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a67651-449c-4e7e-9190-06a89eabf23d None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:37.135763 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a67651-449c-4e7e-9190-06a89eabf23d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:37.141264 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-90a67651-449c-4e7e-9190-06a89eabf23d None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:37.142559 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-90a67651-449c-4e7e-9190-06a89eabf23d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:37.150723 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1181/2375] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:48:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:48:37.154973 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a67651-449c-4e7e-9190-06a89eabf23d None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:37.164660 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a67651-449c-4e7e-9190-06a89eabf23d None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:37.170541 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955add8-1d07-40e7-b93f-7556c02363a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:37.171308 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955add8-1d07-40e7-b93f-7556c02363a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:37.171680 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955add8-1d07-40e7-b93f-7556c02363a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:37.172150 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955add8-1d07-40e7-b93f-7556c02363a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:37.172963 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4955add8-1d07-40e7-b93f-7556c02363a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:48:37.173985 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4955add8-1d07-40e7-b93f-7556c02363a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:37.175711 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4955add8-1d07-40e7-b93f-7556c02363a5 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:37.195685 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1182/2376] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:48:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:48:37.195875 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1195/2377] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:48:36 2022] POST /identity/v3/users => generated 290 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:48:37.195980 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955add8-1d07-40e7-b93f-7556c02363a5 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:37.198107 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb467e-7226-4eea-965c-fb07a297c2da admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:37.199782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-bbdb467e-7226-4eea-965c-fb07a297c2da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:37.206992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955add8-1d07-40e7-b93f-7556c02363a5 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:37.211694 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3156827e-f386-46c3-8f27-375ce2d34ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:37.212353 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156827e-f386-46c3-8f27-375ce2d34ffa admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:37.212699 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156827e-f386-46c3-8f27-375ce2d34ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:37.213124 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156827e-f386-46c3-8f27-375ce2d34ffa admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:37.213880 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3156827e-f386-46c3-8f27-375ce2d34ffa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:37.215187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb467e-7226-4eea-965c-fb07a297c2da admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:37.217970 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3156827e-f386-46c3-8f27-375ce2d34ffa admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:37.220268 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eaf66d-e709-42d8-b2e8-731ec118878a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:37.220660 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eaf66d-e709-42d8-b2e8-731ec118878a admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:37.220938 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eaf66d-e709-42d8-b2e8-731ec118878a admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:37.221217 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eaf66d-e709-42d8-b2e8-731ec118878a admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:37.221866 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eaf66d-e709-42d8-b2e8-731ec118878a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:37.223496 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eaf66d-e709-42d8-b2e8-731ec118878a admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:37.236698 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1196/2378] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:48:37 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:48:37.241396 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eaf66d-e709-42d8-b2e8-731ec118878a admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:37.252630 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eaf66d-e709-42d8-b2e8-731ec118878a admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:37.258015 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf74e6a-ebf2-49de-a560-146135de9d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:37.258638 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf74e6a-ebf2-49de-a560-146135de9d69 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:37.258903 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf74e6a-ebf2-49de-a560-146135de9d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:37.259176 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf74e6a-ebf2-49de-a560-146135de9d69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:37.259978 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf74e6a-ebf2-49de-a560-146135de9d69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:37.261595 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf74e6a-ebf2-49de-a560-146135de9d69 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:37.274249 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1197/2379] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:48:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:48:37.278318 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf74e6a-ebf2-49de-a560-146135de9d69 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:37.288584 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf74e6a-ebf2-49de-a560-146135de9d69 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:37.293716 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd63cc89-b45c-4a10-a420-157e6da89bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:37.294475 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63cc89-b45c-4a10-a420-157e6da89bcf admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:37.295414 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63cc89-b45c-4a10-a420-157e6da89bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:37.295914 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63cc89-b45c-4a10-a420-157e6da89bcf admin admin] PATH_INFO: `/v3/projects/8e8dbc250d3447ecbd8c7b6df2d10d2c/users/b92925514b544fbe9e34326f9c48cb24/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:37.314427 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd63cc89-b45c-4a10-a420-157e6da89bcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e8dbc250d3447ecbd8c7b6df2d10d2c, user_id=b92925514b544fbe9e34326f9c48cb24, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:37.317441 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd63cc89-b45c-4a10-a420-157e6da89bcf admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:37.337657 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1183/2380] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:48:37 2022] POST /identity/v3/projects => generated 363 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:48:37.344118 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3156827e-f386-46c3-8f27-375ce2d34ffa admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:37.377967 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:48:37.410080 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3156827e-f386-46c3-8f27-375ce2d34ffa admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:37.414865 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2243c5b-40e3-4b1a-b35c-ed8a33154ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:37.416439 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2243c5b-40e3-4b1a-b35c-ed8a33154ca1 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:37.416439 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2243c5b-40e3-4b1a-b35c-ed8a33154ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:37.416439 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2243c5b-40e3-4b1a-b35c-ed8a33154ca1 admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:37.416697 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2243c5b-40e3-4b1a-b35c-ed8a33154ca1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:37.418882 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2243c5b-40e3-4b1a-b35c-ed8a33154ca1 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:37.501930 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1198/2381] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:48:37 2022] PUT /identity/v3/projects/8e8dbc250d3447ecbd8c7b6df2d10d2c/users/b92925514b544fbe9e34326f9c48cb24/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:48:37.507040 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc018ac7-5c10-447c-984e-9220d3ddb059 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:37.507323 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc018ac7-5c10-447c-984e-9220d3ddb059 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:37.507603 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc018ac7-5c10-447c-984e-9220d3ddb059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:37.527654 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-fc018ac7-5c10-447c-984e-9220d3ddb059 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:37.528898 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-fc018ac7-5c10-447c-984e-9220d3ddb059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:37.580813 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1184/2382] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:48:37 2022] POST /identity/v3/users => generated 282 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:48:37.585415 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2243c5b-40e3-4b1a-b35c-ed8a33154ca1 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:37.597618 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1199/2383] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:48:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:48:37.600226 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2243c5b-40e3-4b1a-b35c-ed8a33154ca1 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:37.606136 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be5f45-3bc6-40e3-97f4-575ebd8d2e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:37.608145 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be5f45-3bc6-40e3-97f4-575ebd8d2e66 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:37.608145 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be5f45-3bc6-40e3-97f4-575ebd8d2e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:37.608145 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be5f45-3bc6-40e3-97f4-575ebd8d2e66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:37.608145 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be5f45-3bc6-40e3-97f4-575ebd8d2e66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:37.609059 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc018ac7-5c10-447c-984e-9220d3ddb059 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:37.610418 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be5f45-3bc6-40e3-97f4-575ebd8d2e66 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:37.611480 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-fc018ac7-5c10-447c-984e-9220d3ddb059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:37.625142 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1185/2384] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:48:37 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:48:37.630674 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be5f45-3bc6-40e3-97f4-575ebd8d2e66 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:37.642304 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be5f45-3bc6-40e3-97f4-575ebd8d2e66 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:37.647593 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc04f351-f774-4d4a-ba80-f636b4305674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:37.648211 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04f351-f774-4d4a-ba80-f636b4305674 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:37.648527 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04f351-f774-4d4a-ba80-f636b4305674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:37.648804 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04f351-f774-4d4a-ba80-f636b4305674 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:37.649454 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc04f351-f774-4d4a-ba80-f636b4305674 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:37.651055 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc04f351-f774-4d4a-ba80-f636b4305674 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:37.665507 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1186/2385] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:48:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:48:37.669681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc04f351-f774-4d4a-ba80-f636b4305674 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:37.680669 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc04f351-f774-4d4a-ba80-f636b4305674 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:37.690388 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d7e5d-4865-4105-bd5a-9fa3c144f1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:37.691076 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d7e5d-4865-4105-bd5a-9fa3c144f1c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:37.691461 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d7e5d-4865-4105-bd5a-9fa3c144f1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:37.692357 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d7e5d-4865-4105-bd5a-9fa3c144f1c9 admin admin] PATH_INFO: `/v3/projects/70311a67cc974017b535f0d80621d921/users/003c63bcc21740c5b3524fff3364cf69/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:37.697577 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc018ac7-5c10-447c-984e-9220d3ddb059 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:37.701352 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccda1b7-6a4e-4462-9410-e319d1c61f10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:37.701913 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccda1b7-6a4e-4462-9410-e319d1c61f10 service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:37.702192 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccda1b7-6a4e-4462-9410-e319d1c61f10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:37.702463 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccda1b7-6a4e-4462-9410-e319d1c61f10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:37.711571 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454d7e5d-4865-4105-bd5a-9fa3c144f1c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70311a67cc974017b535f0d80621d921, user_id=003c63bcc21740c5b3524fff3364cf69, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:37.714312 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454d7e5d-4865-4105-bd5a-9fa3c144f1c9 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:37.715534 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccda1b7-6a4e-4462-9410-e319d1c61f10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:37.717158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccda1b7-6a4e-4462-9410-e319d1c61f10 service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:37.728033 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1200/2386] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:48:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:48:37.800293 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1187/2387] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:48:37 2022] PUT /identity/v3/projects/70311a67cc974017b535f0d80621d921/users/003c63bcc21740c5b3524fff3364cf69/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:48:37.868945 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5a08e-c7ca-4530-bb15-01ac25af5b0a None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:37.869230 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5a08e-c7ca-4530-bb15-01ac25af5b0a None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:37.869501 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5a08e-c7ca-4530-bb15-01ac25af5b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:37.898795 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-94f5a08e-c7ca-4530-bb15-01ac25af5b0a None None] MFA Rules not processed for user `003c63bcc21740c5b3524fff3364cf69`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:37.900007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-94f5a08e-c7ca-4530-bb15-01ac25af5b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:37.934974 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:48:37.973754 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1201/2388] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:48:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:48:37.987186 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d7e5d-4865-4105-bd5a-9fa3c144f1c9 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:37.994756 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-454d7e5d-4865-4105-bd5a-9fa3c144f1c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:38.061505 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d7e5d-4865-4105-bd5a-9fa3c144f1c9 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:38.065726 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a61e578-eb4f-4f90-acad-f8ad5da4fb32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:38.066323 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a61e578-eb4f-4f90-acad-f8ad5da4fb32 service cinder] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:38.066605 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a61e578-eb4f-4f90-acad-f8ad5da4fb32 service cinder] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:38.066879 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a61e578-eb4f-4f90-acad-f8ad5da4fb32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:38.075675 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a61e578-eb4f-4f90-acad-f8ad5da4fb32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:38.077187 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a61e578-eb4f-4f90-acad-f8ad5da4fb32 service cinder] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:38.086748 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1188/2389] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:48:37 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:48:44.614930 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddbe4ae-aeea-4864-a1bf-f2eb8121a664 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:44.615533 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddbe4ae-aeea-4864-a1bf-f2eb8121a664 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:44.615533 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddbe4ae-aeea-4864-a1bf-f2eb8121a664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:44.667900 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-2ddbe4ae-aeea-4864-a1bf-f2eb8121a664 None None] MFA Rules not processed for user `b92925514b544fbe9e34326f9c48cb24`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:44.669591 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-2ddbe4ae-aeea-4864-a1bf-f2eb8121a664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:44.758644 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1202/2390] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:48:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:48:44.762201 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a61e578-eb4f-4f90-acad-f8ad5da4fb32 service cinder] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:44.848914 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a61e578-eb4f-4f90-acad-f8ad5da4fb32 service cinder] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:44.853997 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bb9c1d-19c5-4876-ae9b-6190804ddcb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:44.854629 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb9c1d-19c5-4876-ae9b-6190804ddcb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:44.854910 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb9c1d-19c5-4876-ae9b-6190804ddcb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:44.855204 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb9c1d-19c5-4876-ae9b-6190804ddcb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:44.855848 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bb9c1d-19c5-4876-ae9b-6190804ddcb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:44.857909 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bb9c1d-19c5-4876-ae9b-6190804ddcb9 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:44.994535 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:48:45.001645 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1189/2391] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:48:44 2022] POST /identity/v3/projects => generated 377 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:48:45.006302 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddbe4ae-aeea-4864-a1bf-f2eb8121a664 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:45.019262 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddbe4ae-aeea-4864-a1bf-f2eb8121a664 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:45.092224 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e0dea5-7a46-475d-94fb-0f7d7f199071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:45.092976 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0dea5-7a46-475d-94fb-0f7d7f199071 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:45.093260 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0dea5-7a46-475d-94fb-0f7d7f199071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:45.093546 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0dea5-7a46-475d-94fb-0f7d7f199071 admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:45.094220 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e0dea5-7a46-475d-94fb-0f7d7f199071 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:45.096387 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e0dea5-7a46-475d-94fb-0f7d7f199071 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:45.228505 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1203/2392] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:48:45 2022] POST /identity/v3/users => generated 295 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:48:45.233310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bb9c1d-19c5-4876-ae9b-6190804ddcb9 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:45.247939 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bb9c1d-19c5-4876-ae9b-6190804ddcb9 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:45.253779 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609efe7d-3863-4b73-bcca-57ef3a229587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:45.254462 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609efe7d-3863-4b73-bcca-57ef3a229587 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:45.254841 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609efe7d-3863-4b73-bcca-57ef3a229587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:45.255434 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609efe7d-3863-4b73-bcca-57ef3a229587 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:45.256414 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609efe7d-3863-4b73-bcca-57ef3a229587 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:45.258348 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609efe7d-3863-4b73-bcca-57ef3a229587 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:45.271547 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1190/2393] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:48:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:48:45.282356 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e0dea5-7a46-475d-94fb-0f7d7f199071 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:45.293077 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e0dea5-7a46-475d-94fb-0f7d7f199071 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:45.298307 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6308b27-c170-4af1-8538-f89973b27ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:45.298983 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6308b27-c170-4af1-8538-f89973b27ff4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:45.299265 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6308b27-c170-4af1-8538-f89973b27ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:45.299545 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6308b27-c170-4af1-8538-f89973b27ff4 admin admin] PATH_INFO: `/v3/projects/2a6d93b55fbc4d6a83b413cbb085f972/users/c3c573cbfb4c4fb0aad5fbbdf7aac245/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:45.316897 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6308b27-c170-4af1-8538-f89973b27ff4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a6d93b55fbc4d6a83b413cbb085f972, user_id=c3c573cbfb4c4fb0aad5fbbdf7aac245, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:45.319720 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6308b27-c170-4af1-8538-f89973b27ff4 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:45.436038 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1204/2394] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:48:45 2022] PUT /identity/v3/projects/2a6d93b55fbc4d6a83b413cbb085f972/users/c3c573cbfb4c4fb0aad5fbbdf7aac245/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:48:45.440114 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609efe7d-3863-4b73-bcca-57ef3a229587 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:45.451172 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609efe7d-3863-4b73-bcca-57ef3a229587 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:45.511800 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a3c33-168e-4e2f-b43d-ac6dbb58c83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:45.512437 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a3c33-168e-4e2f-b43d-ac6dbb58c83e admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:45.512729 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a3c33-168e-4e2f-b43d-ac6dbb58c83e admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:45.513008 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a3c33-168e-4e2f-b43d-ac6dbb58c83e admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:45.513658 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338a3c33-168e-4e2f-b43d-ac6dbb58c83e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:45.515321 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338a3c33-168e-4e2f-b43d-ac6dbb58c83e admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:45.525781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1191/2395] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:48:45 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:48:45.531629 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6308b27-c170-4af1-8538-f89973b27ff4 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:45.541385 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6308b27-c170-4af1-8538-f89973b27ff4 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:45.546404 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f3b9f-b70e-471b-8429-a0d6f88fdc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:45.547059 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f3b9f-b70e-471b-8429-a0d6f88fdc96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:45.547346 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f3b9f-b70e-471b-8429-a0d6f88fdc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:45.547624 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f3b9f-b70e-471b-8429-a0d6f88fdc96 admin admin] PATH_INFO: `/v3/projects/2a6d93b55fbc4d6a83b413cbb085f972/users/c3c573cbfb4c4fb0aad5fbbdf7aac245/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:45.549625 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470f3b9f-b70e-471b-8429-a0d6f88fdc96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a6d93b55fbc4d6a83b413cbb085f972, user_id=c3c573cbfb4c4fb0aad5fbbdf7aac245, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:45.553201 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470f3b9f-b70e-471b-8429-a0d6f88fdc96 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:45.563660 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1205/2396] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:48:45 2022] PUT /identity/v3/projects/2a6d93b55fbc4d6a83b413cbb085f972/users/c3c573cbfb4c4fb0aad5fbbdf7aac245/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:48:52.823770 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a57f0-4156-4793-b300-bf6da6565cf9 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:52.824295 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a57f0-4156-4793-b300-bf6da6565cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:52.824295 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a57f0-4156-4793-b300-bf6da6565cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:52.862501 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-c05a57f0-4156-4793-b300-bf6da6565cf9 None None] MFA Rules not processed for user `c3c573cbfb4c4fb0aad5fbbdf7aac245`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:52.863980 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-c05a57f0-4156-4793-b300-bf6da6565cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:53.008568 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1192/2397] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:48:52 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:48:53.022075 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f3b9f-b70e-471b-8429-a0d6f88fdc96 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:53.024690 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-470f3b9f-b70e-471b-8429-a0d6f88fdc96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:53.046919 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:48:53.142203 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f3b9f-b70e-471b-8429-a0d6f88fdc96 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:53.149280 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7452a282-0cbf-47e7-b787-a19dd3554e49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:53.150581 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452a282-0cbf-47e7-b787-a19dd3554e49 service nova] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:53.150581 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452a282-0cbf-47e7-b787-a19dd3554e49 service nova] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:53.150852 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452a282-0cbf-47e7-b787-a19dd3554e49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:53.226598 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7452a282-0cbf-47e7-b787-a19dd3554e49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:53.228637 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7452a282-0cbf-47e7-b787-a19dd3554e49 service nova] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:53.243780 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1206/2398] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:48:53 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:48:54.466733 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05a57f0-4156-4793-b300-bf6da6565cf9 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:54.468281 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-c05a57f0-4156-4793-b300-bf6da6565cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:54.545682 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05a57f0-4156-4793-b300-bf6da6565cf9 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:54.550398 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99897fdf-15bd-46de-87c3-ab2300a9fc67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:54.555530 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99897fdf-15bd-46de-87c3-ab2300a9fc67 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:54.555530 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99897fdf-15bd-46de-87c3-ab2300a9fc67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:54.555530 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99897fdf-15bd-46de-87c3-ab2300a9fc67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:54.555530 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-99897fdf-15bd-46de-87c3-ab2300a9fc67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:54.582143 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:48:54.624106 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99897fdf-15bd-46de-87c3-ab2300a9fc67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:54.625684 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99897fdf-15bd-46de-87c3-ab2300a9fc67 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:54.636972 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1193/2399] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:48:54 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:48:59.417996 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450a67d-3525-440f-ab54-10bb1e9313d5 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:59.418524 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450a67d-3525-440f-ab54-10bb1e9313d5 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:59.418524 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450a67d-3525-440f-ab54-10bb1e9313d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:59.439814 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-8450a67d-3525-440f-ab54-10bb1e9313d5 None None] MFA Rules not processed for user `604380c191c8461bbb937b8d951238bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:59.441605 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-8450a67d-3525-440f-ab54-10bb1e9313d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:59.503147 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1207/2400] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Aug 24 05:48:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:48:59.731428 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99897fdf-15bd-46de-87c3-ab2300a9fc67 service neutron] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:59.732975 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-99897fdf-15bd-46de-87c3-ab2300a9fc67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:59.808903 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99897fdf-15bd-46de-87c3-ab2300a9fc67 service neutron] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:59.814884 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f5c86-8581-4f8a-976e-8bc3e73ed149 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb2b6cae4a004539a95fc7dd558db255', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:59.817143 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f5c86-8581-4f8a-976e-8bc3e73ed149 service swift] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:59.817143 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f5c86-8581-4f8a-976e-8bc3e73ed149 service swift] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:59.817143 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f5c86-8581-4f8a-976e-8bc3e73ed149 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:59.826919 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6f5c86-8581-4f8a-976e-8bc3e73ed149 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:59.828602 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6f5c86-8581-4f8a-976e-8bc3e73ed149 service swift] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:59.839203 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1194/2401] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Aug 24 05:48:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 05:49:01.092424 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2967bf6a-a583-4901-a044-ef757c4b2d51 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:49:01.092927 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2967bf6a-a583-4901-a044-ef757c4b2d51 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:49:01.094978 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2967bf6a-a583-4901-a044-ef757c4b2d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:49:01.112939 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-2967bf6a-a583-4901-a044-ef757c4b2d51 None None] MFA Rules not processed for user `604380c191c8461bbb937b8d951238bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:49:01.114174 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-2967bf6a-a583-4901-a044-ef757c4b2d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:49:01.121601 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1208/2402] 10.222.0.15 () {58 vars in 967 bytes} [Wed Aug 24 05:49:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:49:01.134428 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f5c86-8581-4f8a-976e-8bc3e73ed149 service swift] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:49:01.146135 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f5c86-8581-4f8a-976e-8bc3e73ed149 service swift] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:49:01.152100 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624890b2-fa88-42a0-8a4c-e9a7e598efe9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb2b6cae4a004539a95fc7dd558db255', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:49:01.152774 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624890b2-fa88-42a0-8a4c-e9a7e598efe9 service swift] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:49:01.153052 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624890b2-fa88-42a0-8a4c-e9a7e598efe9 service swift] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:49:01.153333 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624890b2-fa88-42a0-8a4c-e9a7e598efe9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:49:01.198154 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624890b2-fa88-42a0-8a4c-e9a7e598efe9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:49:01.199939 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624890b2-fa88-42a0-8a4c-e9a7e598efe9 service swift] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:49:01.223808 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1195/2403] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Aug 24 05:49:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 05:49:02.403765 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645cf315-0b37-46f4-8f7d-21897a1626bb None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:49:02.403765 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645cf315-0b37-46f4-8f7d-21897a1626bb None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:49:02.404326 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645cf315-0b37-46f4-8f7d-21897a1626bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:49:02.426051 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-645cf315-0b37-46f4-8f7d-21897a1626bb None None] MFA Rules not processed for user `604380c191c8461bbb937b8d951238bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:49:02.427387 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-645cf315-0b37-46f4-8f7d-21897a1626bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:49:02.435325 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1209/2404] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Aug 24 05:49:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:49:02.451973 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624890b2-fa88-42a0-8a4c-e9a7e598efe9 service swift] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:49:02.468144 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624890b2-fa88-42a0-8a4c-e9a7e598efe9 service swift] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:49:02.473969 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7583dd3e-6fcd-4be3-8b5f-76b28c0fbb8f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb2b6cae4a004539a95fc7dd558db255', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:49:02.474579 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583dd3e-6fcd-4be3-8b5f-76b28c0fbb8f service swift] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:49:02.474860 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583dd3e-6fcd-4be3-8b5f-76b28c0fbb8f service swift] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:49:02.475136 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583dd3e-6fcd-4be3-8b5f-76b28c0fbb8f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:49:02.483442 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7583dd3e-6fcd-4be3-8b5f-76b28c0fbb8f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:49:02.485065 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7583dd3e-6fcd-4be3-8b5f-76b28c0fbb8f service swift] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:49:02.494931 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1196/2405] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Aug 24 05:49:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 05:49:03.394772 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a27d0-dcbe-40a0-a2fc-8c429cc709cc None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:49:03.395070 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a27d0-dcbe-40a0-a2fc-8c429cc709cc None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:49:03.395533 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a27d0-dcbe-40a0-a2fc-8c429cc709cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:49:03.416127 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-a32a27d0-dcbe-40a0-a2fc-8c429cc709cc None None] MFA Rules not processed for user `604380c191c8461bbb937b8d951238bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:49:03.418252 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-a32a27d0-dcbe-40a0-a2fc-8c429cc709cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:49:03.436722 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1210/2406] 10.222.0.15 () {58 vars in 967 bytes} [Wed Aug 24 05:49:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:49:03.448577 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7583dd3e-6fcd-4be3-8b5f-76b28c0fbb8f service swift] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:49:03.460745 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7583dd3e-6fcd-4be3-8b5f-76b28c0fbb8f service swift] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:49:03.466024 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38fe21f-303a-4e5e-8760-60e3d8599489 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb2b6cae4a004539a95fc7dd558db255', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:49:03.466640 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38fe21f-303a-4e5e-8760-60e3d8599489 service swift] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:49:03.466919 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38fe21f-303a-4e5e-8760-60e3d8599489 service swift] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:49:03.467191 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38fe21f-303a-4e5e-8760-60e3d8599489 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:49:03.478422 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38fe21f-303a-4e5e-8760-60e3d8599489 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:49:03.481524 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38fe21f-303a-4e5e-8760-60e3d8599489 service swift] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:49:03.497003 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1197/2407] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Aug 24 05:49:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 05:49:06.565070 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a27d0-dcbe-40a0-a2fc-8c429cc709cc None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:49:06.567232 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-a32a27d0-dcbe-40a0-a2fc-8c429cc709cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:49:06.613452 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=100 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:49:06.642595 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a27d0-dcbe-40a0-a2fc-8c429cc709cc None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:49:06.648639 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56cfa0c-202c-4635-9122-7042217fc019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:49:06.649278 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56cfa0c-202c-4635-9122-7042217fc019 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:49:06.649566 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56cfa0c-202c-4635-9122-7042217fc019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:49:06.649909 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56cfa0c-202c-4635-9122-7042217fc019 admin admin] PATH_INFO: `/v3/users/26bdb5e1c6d145e8a8c6792b2534a836` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:49:06.651085 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56cfa0c-202c-4635-9122-7042217fc019 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26bdb5e1c6d145e8a8c6792b2534a836)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:49:06.653071 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56cfa0c-202c-4635-9122-7042217fc019 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:49:06.928347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-d56cfa0c-202c-4635-9122-7042217fc019 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26bdb5e1c6d145e8a8c6792b2534a836'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:49:07.006990 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-d56cfa0c-202c-4635-9122-7042217fc019 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26bdb5e1c6d145e8a8c6792b2534a836'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:49:07.014994 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-d56cfa0c-202c-4635-9122-7042217fc019 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26bdb5e1c6d145e8a8c6792b2534a836'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:49:07.031450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-d56cfa0c-202c-4635-9122-7042217fc019 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26bdb5e1c6d145e8a8c6792b2534a836'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:49:07.038304 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-d56cfa0c-202c-4635-9122-7042217fc019 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26bdb5e1c6d145e8a8c6792b2534a836'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:49:07.045148 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1211/2408] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:49:06 2022] DELETE /identity/v3/users/26bdb5e1c6d145e8a8c6792b2534a836 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:49:07.049769 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2144211-8e8b-4c57-bd0c-ec7772b2732a None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:49:07.050240 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2144211-8e8b-4c57-bd0c-ec7772b2732a None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:49:07.050336 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2144211-8e8b-4c57-bd0c-ec7772b2732a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:49:07.070441 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-a2144211-8e8b-4c57-bd0c-ec7772b2732a None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:49:07.071639 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-a2144211-8e8b-4c57-bd0c-ec7772b2732a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:49:07.138292 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1198/2409] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:49:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:49:07.148085 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56cfa0c-202c-4635-9122-7042217fc019 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:49:07.149567 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-d56cfa0c-202c-4635-9122-7042217fc019 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:49:07.214102 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56cfa0c-202c-4635-9122-7042217fc019 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:49:07.217984 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185a5077-4d31-45e3-a9b3-751f1dd5bb0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:49:07.218632 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a5077-4d31-45e3-a9b3-751f1dd5bb0d service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:49:07.218894 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a5077-4d31-45e3-a9b3-751f1dd5bb0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:49:07.219170 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a5077-4d31-45e3-a9b3-751f1dd5bb0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:49:07.228942 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185a5077-4d31-45e3-a9b3-751f1dd5bb0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:49:07.230416 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185a5077-4d31-45e3-a9b3-751f1dd5bb0d service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:49:07.241410 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1212/2410] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:49:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:49:07.582927 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2144211-8e8b-4c57-bd0c-ec7772b2732a None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:49:07.585489 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-a2144211-8e8b-4c57-bd0c-ec7772b2732a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:49:07.600896 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2144211-8e8b-4c57-bd0c-ec7772b2732a None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:49:07.606540 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578cbe3f-0a5c-416e-b273-bee74bd84f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:49:07.607550 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578cbe3f-0a5c-416e-b273-bee74bd84f89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:49:07.607933 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578cbe3f-0a5c-416e-b273-bee74bd84f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:49:07.608262 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578cbe3f-0a5c-416e-b273-bee74bd84f89 admin admin] PATH_INFO: `/v3/projects/53e1b207a1e9479e88e00371843f7ddd` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:49:07.609400 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578cbe3f-0a5c-416e-b273-bee74bd84f89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53e1b207a1e9479e88e00371843f7ddd)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:49:07.611473 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578cbe3f-0a5c-416e-b273-bee74bd84f89 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:49:07.622590 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:49:07.798363 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-578cbe3f-0a5c-416e-b273-bee74bd84f89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53e1b207a1e9479e88e00371843f7ddd', 'request_id': 'req-578cbe3f-0a5c-416e-b273-bee74bd84f89'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:49:07.805648 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-578cbe3f-0a5c-416e-b273-bee74bd84f89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53e1b207a1e9479e88e00371843f7ddd', 'request_id': 'req-578cbe3f-0a5c-416e-b273-bee74bd84f89'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:49:07.910544 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-578cbe3f-0a5c-416e-b273-bee74bd84f89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53e1b207a1e9479e88e00371843f7ddd', 'request_id': 'req-578cbe3f-0a5c-416e-b273-bee74bd84f89'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:49:07.921945 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-578cbe3f-0a5c-416e-b273-bee74bd84f89 admin admin] The token cache is being invalidate because project 53e1b207a1e9479e88e00371843f7ddd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:49:07.922614 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-578cbe3f-0a5c-416e-b273-bee74bd84f89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:49:07.928023 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-578cbe3f-0a5c-416e-b273-bee74bd84f89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:49:07.928023 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1199/2411] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:49:07 2022] DELETE /identity/v3/projects/53e1b207a1e9479e88e00371843f7ddd => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:49:10.601289 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91e9bd-6bbb-4fda-87db-de1fa02fcf60 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:49:10.601902 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91e9bd-6bbb-4fda-87db-de1fa02fcf60 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:49:10.601902 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91e9bd-6bbb-4fda-87db-de1fa02fcf60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:49:10.626495 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-8a91e9bd-6bbb-4fda-87db-de1fa02fcf60 None None] MFA Rules not processed for user `d3e6e20d5cd44444b43b1e7abb240f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:49:10.628084 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-8a91e9bd-6bbb-4fda-87db-de1fa02fcf60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:49:10.696153 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1213/2412] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:49:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:49:17.918497 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:49:20.688746 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:49:23.178089 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578cbe3f-0a5c-416e-b273-bee74bd84f89 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:49:23.179959 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-578cbe3f-0a5c-416e-b273-bee74bd84f89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:49:23.264797 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578cbe3f-0a5c-416e-b273-bee74bd84f89 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:49:23.268896 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8af802-9cc6-4a3f-a91e-5c3e3ddae240 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:49:23.269673 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8af802-9cc6-4a3f-a91e-5c3e3ddae240 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:49:23.270037 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8af802-9cc6-4a3f-a91e-5c3e3ddae240 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:49:23.270356 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8af802-9cc6-4a3f-a91e-5c3e3ddae240 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:49:23.272322 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-cc8af802-9cc6-4a3f-a91e-5c3e3ddae240 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:49:23.358644 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8af802-9cc6-4a3f-a91e-5c3e3ddae240 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:49:23.360102 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8af802-9cc6-4a3f-a91e-5c3e3ddae240 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:49:23.374061 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1200/2413] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:49:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:49:25.729422 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91e9bd-6bbb-4fda-87db-de1fa02fcf60 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:49:25.740641 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91e9bd-6bbb-4fda-87db-de1fa02fcf60 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:49:25.744810 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fcdddb-ba6d-4007-8436-09c4e808ef93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:49:25.745456 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fcdddb-ba6d-4007-8436-09c4e808ef93 service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:49:25.745853 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fcdddb-ba6d-4007-8436-09c4e808ef93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:49:25.746270 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fcdddb-ba6d-4007-8436-09c4e808ef93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:49:25.748675 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-f8fcdddb-ba6d-4007-8436-09c4e808ef93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:49:25.764858 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fcdddb-ba6d-4007-8436-09c4e808ef93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:49:25.767894 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fcdddb-ba6d-4007-8436-09c4e808ef93 service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:49:25.779286 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1214/2414] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:49:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:49:33.368102 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=54 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:49:35.773508 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=6 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:49:36.108457 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3551e13e-de47-440b-9b4d-b73d3ed38422 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:49:36.108996 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3551e13e-de47-440b-9b4d-b73d3ed38422 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:49:36.108996 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3551e13e-de47-440b-9b4d-b73d3ed38422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:49:36.129392 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-3551e13e-de47-440b-9b4d-b73d3ed38422 None None] MFA Rules not processed for user `b92925514b544fbe9e34326f9c48cb24`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:49:36.130537 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-3551e13e-de47-440b-9b4d-b73d3ed38422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:49:36.206761 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1201/2415] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:49:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:49:36.665267 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fcdddb-ba6d-4007-8436-09c4e808ef93 service neutron] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:49:36.666928 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-f8fcdddb-ba6d-4007-8436-09c4e808ef93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:49:36.682087 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fcdddb-ba6d-4007-8436-09c4e808ef93 service neutron] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:49:36.688569 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7915f0f6-c2a5-4bc6-9a92-e8b46a0068ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:49:36.689200 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7915f0f6-c2a5-4bc6-9a92-e8b46a0068ae service nova] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:49:36.689488 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7915f0f6-c2a5-4bc6-9a92-e8b46a0068ae service nova] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:49:36.689777 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7915f0f6-c2a5-4bc6-9a92-e8b46a0068ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:49:36.691710 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-7915f0f6-c2a5-4bc6-9a92-e8b46a0068ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:49:36.703373 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7915f0f6-c2a5-4bc6-9a92-e8b46a0068ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:49:36.705034 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7915f0f6-c2a5-4bc6-9a92-e8b46a0068ae service nova] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:49:36.715037 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1215/2416] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:49:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:49:46.197876 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=25 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:49:46.711804 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=6 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:49:48.362856 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3551e13e-de47-440b-9b4d-b73d3ed38422 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:49:48.364459 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-3551e13e-de47-440b-9b4d-b73d3ed38422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:49:48.435412 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3551e13e-de47-440b-9b4d-b73d3ed38422 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:49:48.441125 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d9c72e-53d4-4665-80c9-91a66878f539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:49:48.441703 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d9c72e-53d4-4665-80c9-91a66878f539 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:49:48.441984 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d9c72e-53d4-4665-80c9-91a66878f539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:49:48.442260 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d9c72e-53d4-4665-80c9-91a66878f539 admin admin] PATH_INFO: `/v3/users/003c63bcc21740c5b3524fff3364cf69` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:49:48.443438 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d9c72e-53d4-4665-80c9-91a66878f539 admin admin] RBAC: Authorizing `identity:delete_user(user_id=003c63bcc21740c5b3524fff3364cf69)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:49:48.445473 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d9c72e-53d4-4665-80c9-91a66878f539 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:49:48.670655 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-56d9c72e-53d4-4665-80c9-91a66878f539 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '003c63bcc21740c5b3524fff3364cf69'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:49:48.679308 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-56d9c72e-53d4-4665-80c9-91a66878f539 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '003c63bcc21740c5b3524fff3364cf69'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:49:48.680024 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-56d9c72e-53d4-4665-80c9-91a66878f539 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '003c63bcc21740c5b3524fff3364cf69'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:49:48.751963 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-56d9c72e-53d4-4665-80c9-91a66878f539 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '003c63bcc21740c5b3524fff3364cf69'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:49:48.766482 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-56d9c72e-53d4-4665-80c9-91a66878f539 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '003c63bcc21740c5b3524fff3364cf69'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:49:48.772953 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1202/2417] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:49:48 2022] DELETE /identity/v3/users/003c63bcc21740c5b3524fff3364cf69 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:49:48.777559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec4e47-b0fe-4f44-9ae0-4927513c89c5 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:49:48.778071 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec4e47-b0fe-4f44-9ae0-4927513c89c5 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:49:48.778172 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec4e47-b0fe-4f44-9ae0-4927513c89c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:49:48.799966 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-63ec4e47-b0fe-4f44-9ae0-4927513c89c5 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:49:48.801289 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-63ec4e47-b0fe-4f44-9ae0-4927513c89c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:49:48.862897 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1216/2418] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:49:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:49:48.872809 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d9c72e-53d4-4665-80c9-91a66878f539 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:49:48.874822 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-56d9c72e-53d4-4665-80c9-91a66878f539 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:49:48.936653 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d9c72e-53d4-4665-80c9-91a66878f539 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:49:48.940525 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf4be2b-f9a0-4704-831b-1b4e6594c06c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:49:48.941146 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4be2b-f9a0-4704-831b-1b4e6594c06c service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:49:48.941432 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4be2b-f9a0-4704-831b-1b4e6594c06c service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:49:48.941709 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4be2b-f9a0-4704-831b-1b4e6594c06c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:49:48.951474 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf4be2b-f9a0-4704-831b-1b4e6594c06c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:49:48.954202 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf4be2b-f9a0-4704-831b-1b4e6594c06c service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:49:48.964078 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1203/2419] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:49:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:49:49.502111 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec4e47-b0fe-4f44-9ae0-4927513c89c5 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:49:49.503589 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-63ec4e47-b0fe-4f44-9ae0-4927513c89c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:49:49.516352 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec4e47-b0fe-4f44-9ae0-4927513c89c5 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:49:49.521215 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59f57d3-cc49-474e-bd95-ced7446df366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:49:49.521870 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f57d3-cc49-474e-bd95-ced7446df366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:49:49.522151 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f57d3-cc49-474e-bd95-ced7446df366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:49:49.522429 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f57d3-cc49-474e-bd95-ced7446df366 admin admin] PATH_INFO: `/v3/projects/70311a67cc974017b535f0d80621d921` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:49:49.523573 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59f57d3-cc49-474e-bd95-ced7446df366 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70311a67cc974017b535f0d80621d921)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:49:49.527696 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59f57d3-cc49-474e-bd95-ced7446df366 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:49:49.651020 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-c59f57d3-cc49-474e-bd95-ced7446df366 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70311a67cc974017b535f0d80621d921', 'request_id': 'req-c59f57d3-cc49-474e-bd95-ced7446df366'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:49:49.658193 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-c59f57d3-cc49-474e-bd95-ced7446df366 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70311a67cc974017b535f0d80621d921', 'request_id': 'req-c59f57d3-cc49-474e-bd95-ced7446df366'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:49:49.740484 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-c59f57d3-cc49-474e-bd95-ced7446df366 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70311a67cc974017b535f0d80621d921', 'request_id': 'req-c59f57d3-cc49-474e-bd95-ced7446df366'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:49:49.752078 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-c59f57d3-cc49-474e-bd95-ced7446df366 admin admin] The token cache is being invalidate because project 70311a67cc974017b535f0d80621d921 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:49:49.752418 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-c59f57d3-cc49-474e-bd95-ced7446df366 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:49:49.753110 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-c59f57d3-cc49-474e-bd95-ced7446df366 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:49:49.754607 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1217/2420] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:49:49 2022] DELETE /identity/v3/projects/70311a67cc974017b535f0d80621d921 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:49:56.810151 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf4be2b-f9a0-4704-831b-1b4e6594c06c service neutron] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:49:56.820180 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf4be2b-f9a0-4704-831b-1b4e6594c06c service neutron] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:49:56.840214 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:49:56.878549 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd34bb3-0c72-4cb3-bb98-3877afdaf811 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:49:56.879175 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd34bb3-0c72-4cb3-bb98-3877afdaf811 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:49:56.879576 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd34bb3-0c72-4cb3-bb98-3877afdaf811 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:49:56.879852 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd34bb3-0c72-4cb3-bb98-3877afdaf811 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:49:56.881983 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-7bd34bb3-0c72-4cb3-bb98-3877afdaf811 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:49:56.893283 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd34bb3-0c72-4cb3-bb98-3877afdaf811 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:49:56.895746 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd34bb3-0c72-4cb3-bb98-3877afdaf811 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:49:56.903604 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1204/2421] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:49:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:49:59.749947 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:50:01.183734 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59f57d3-cc49-474e-bd95-ced7446df366 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:50:01.185381 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-c59f57d3-cc49-474e-bd95-ced7446df366 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:50:01.197677 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59f57d3-cc49-474e-bd95-ced7446df366 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:50:01.201440 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3303aa9-cd90-448a-866f-d1ba274afe68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:50:01.202211 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3303aa9-cd90-448a-866f-d1ba274afe68 service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:50:01.202752 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3303aa9-cd90-448a-866f-d1ba274afe68 service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:50:01.202752 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3303aa9-cd90-448a-866f-d1ba274afe68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:50:01.204484 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-f3303aa9-cd90-448a-866f-d1ba274afe68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:50:01.269959 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3303aa9-cd90-448a-866f-d1ba274afe68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:50:01.271488 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3303aa9-cd90-448a-866f-d1ba274afe68 service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:50:01.281606 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1218/2422] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:50:01 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:50:03.374811 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd34bb3-0c72-4cb3-bb98-3877afdaf811 service neutron] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:50:03.376351 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-7bd34bb3-0c72-4cb3-bb98-3877afdaf811 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:50:03.473587 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd34bb3-0c72-4cb3-bb98-3877afdaf811 service neutron] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:50:03.479335 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4131dcca-1a34-4eef-a941-2804e0e434f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:50:03.480453 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4131dcca-1a34-4eef-a941-2804e0e434f0 service nova] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:50:03.480453 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4131dcca-1a34-4eef-a941-2804e0e434f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:50:03.480639 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4131dcca-1a34-4eef-a941-2804e0e434f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:50:03.482949 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-4131dcca-1a34-4eef-a941-2804e0e434f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:50:03.497957 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4131dcca-1a34-4eef-a941-2804e0e434f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:50:03.500427 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4131dcca-1a34-4eef-a941-2804e0e434f0 service nova] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:50:03.512328 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1205/2423] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:50:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:50:11.276908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=30 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:50:13.507296 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=50 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:50:28.418462 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3303aa9-cd90-448a-866f-d1ba274afe68 service neutron] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:50:28.431793 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3303aa9-cd90-448a-866f-d1ba274afe68 service neutron] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:50:28.437407 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f71ab-3c74-465c-a6a4-93cea41a9294 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:50:28.438020 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f71ab-3c74-465c-a6a4-93cea41a9294 service nova] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:50:28.438305 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f71ab-3c74-465c-a6a4-93cea41a9294 service nova] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:50:28.438585 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f71ab-3c74-465c-a6a4-93cea41a9294 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:50:28.441498 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-7b7f71ab-3c74-465c-a6a4-93cea41a9294 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:50:28.453899 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7f71ab-3c74-465c-a6a4-93cea41a9294 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:50:28.456055 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7f71ab-3c74-465c-a6a4-93cea41a9294 service nova] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:50:28.466744 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1219/2424] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:50:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:50:32.190113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89aa0e-4ea5-4042-b279-ec34b6fff6cd None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:50:32.190538 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89aa0e-4ea5-4042-b279-ec34b6fff6cd None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:50:32.190672 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89aa0e-4ea5-4042-b279-ec34b6fff6cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:50:32.210269 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-ed89aa0e-4ea5-4042-b279-ec34b6fff6cd None None] MFA Rules not processed for user `604380c191c8461bbb937b8d951238bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:50:32.211469 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-ed89aa0e-4ea5-4042-b279-ec34b6fff6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:50:32.268648 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1206/2425] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Aug 24 05:50:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:50:32.419133 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f71ab-3c74-465c-a6a4-93cea41a9294 service nova] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:50:32.420875 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-7b7f71ab-3c74-465c-a6a4-93cea41a9294 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:50:32.493667 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f71ab-3c74-465c-a6a4-93cea41a9294 service nova] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:50:32.498910 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32484143-9080-484d-b231-f6489bf4bc76 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb2b6cae4a004539a95fc7dd558db255', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:50:32.499532 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32484143-9080-484d-b231-f6489bf4bc76 service swift] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:50:32.499813 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32484143-9080-484d-b231-f6489bf4bc76 service swift] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:50:32.500088 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32484143-9080-484d-b231-f6489bf4bc76 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:50:32.509427 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32484143-9080-484d-b231-f6489bf4bc76 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:50:32.511139 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32484143-9080-484d-b231-f6489bf4bc76 service swift] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:50:32.520654 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1220/2426] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Aug 24 05:50:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 05:50:41.274547 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89aa0e-4ea5-4042-b279-ec34b6fff6cd None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:50:41.302163 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89aa0e-4ea5-4042-b279-ec34b6fff6cd None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:50:41.313543 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772232d5-7b66-4427-9139-212c88f03aa8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:50:41.314172 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772232d5-7b66-4427-9139-212c88f03aa8 service nova] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:50:41.314459 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772232d5-7b66-4427-9139-212c88f03aa8 service nova] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:50:41.314741 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772232d5-7b66-4427-9139-212c88f03aa8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:50:41.317825 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-772232d5-7b66-4427-9139-212c88f03aa8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:50:41.381798 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772232d5-7b66-4427-9139-212c88f03aa8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:50:41.383426 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772232d5-7b66-4427-9139-212c88f03aa8 service nova] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:50:41.396491 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1207/2427] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:50:41 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:50:42.516203 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=38 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:50:51.389264 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=53 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:50:54.580244 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32484143-9080-484d-b231-f6489bf4bc76 service swift] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:50:54.581831 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-32484143-9080-484d-b231-f6489bf4bc76 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:50:54.594673 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32484143-9080-484d-b231-f6489bf4bc76 service swift] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:50:54.598317 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf251ac7-77d9-493c-90d3-0e71d12e7f63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:50:54.598916 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf251ac7-77d9-493c-90d3-0e71d12e7f63 service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:50:54.599196 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf251ac7-77d9-493c-90d3-0e71d12e7f63 service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:50:54.599479 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf251ac7-77d9-493c-90d3-0e71d12e7f63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:50:54.601918 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-bf251ac7-77d9-493c-90d3-0e71d12e7f63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:50:54.611435 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf251ac7-77d9-493c-90d3-0e71d12e7f63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:50:54.612893 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf251ac7-77d9-493c-90d3-0e71d12e7f63 service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:50:54.621133 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1221/2428] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:50:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:50:55.080234 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacde86f-fb84-4de0-bb1c-945cd42f7b33 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:50:55.080781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacde86f-fb84-4de0-bb1c-945cd42f7b33 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:50:55.080918 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacde86f-fb84-4de0-bb1c-945cd42f7b33 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:50:55.082163 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1208/2429] 10.222.0.15 () {54 vars in 921 bytes} [Wed Aug 24 05:50:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:50:55.088105 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff880c3-5eaa-4222-b9a1-1891bc9773ff None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:50:55.088662 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff880c3-5eaa-4222-b9a1-1891bc9773ff None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:50:55.089014 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff880c3-5eaa-4222-b9a1-1891bc9773ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:50:55.109499 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-fff880c3-5eaa-4222-b9a1-1891bc9773ff None None] MFA Rules not processed for user `fbecffb800584c5786813accd061421b`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:50:55.110795 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-fff880c3-5eaa-4222-b9a1-1891bc9773ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:50:55.121061 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1222/2430] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Aug 24 05:50:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:50:55.132948 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacde86f-fb84-4de0-bb1c-945cd42f7b33 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:50:55.134703 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-eacde86f-fb84-4de0-bb1c-945cd42f7b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:50:55.200232 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacde86f-fb84-4de0-bb1c-945cd42f7b33 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:50:55.204118 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ea6307-53c6-444f-86dc-409f2d693567 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9aee9c524a974a248c3bf363444f584a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:50:55.204752 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea6307-53c6-444f-86dc-409f2d693567 service glance] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:50:55.205032 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea6307-53c6-444f-86dc-409f2d693567 service glance] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:50:55.205307 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea6307-53c6-444f-86dc-409f2d693567 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:50:55.215462 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ea6307-53c6-444f-86dc-409f2d693567 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:50:55.216919 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ea6307-53c6-444f-86dc-409f2d693567 service glance] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:50:55.228367 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1209/2431] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:50:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:50:55.413131 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72edc34-ecce-41d6-82bd-5bdccd38c0d2 None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:50:55.413415 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72edc34-ecce-41d6-82bd-5bdccd38c0d2 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:50:55.413696 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72edc34-ecce-41d6-82bd-5bdccd38c0d2 None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:50:55.415085 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1223/2432] 10.222.0.15 () {54 vars in 921 bytes} [Wed Aug 24 05:50:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:50:55.419761 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa990b4b-415d-4ddc-9c02-6bf6804211e3 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:50:55.420054 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa990b4b-415d-4ddc-9c02-6bf6804211e3 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:50:55.420667 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa990b4b-415d-4ddc-9c02-6bf6804211e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:50:55.440869 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-aa990b4b-415d-4ddc-9c02-6bf6804211e3 None None] MFA Rules not processed for user `b1ecd9b302b64ecf81d7e3d3356ddf7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:50:55.442063 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-aa990b4b-415d-4ddc-9c02-6bf6804211e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:50:55.449165 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1210/2433] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Aug 24 05:50:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:50:55.461400 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72edc34-ecce-41d6-82bd-5bdccd38c0d2 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:50:55.472454 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72edc34-ecce-41d6-82bd-5bdccd38c0d2 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:50:55.477313 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde054a4-de66-4f0d-8875-10a22e502cf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:50:55.478182 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde054a4-de66-4f0d-8875-10a22e502cf6 service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:50:55.478559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde054a4-de66-4f0d-8875-10a22e502cf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:50:55.478928 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde054a4-de66-4f0d-8875-10a22e502cf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:50:55.486375 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde054a4-de66-4f0d-8875-10a22e502cf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:50:55.487838 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde054a4-de66-4f0d-8875-10a22e502cf6 service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:50:55.495622 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1224/2434] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:50:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:50:55.581320 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b156ff-3d5f-43aa-a2cd-92087e67200b None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:50:55.581634 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b156ff-3d5f-43aa-a2cd-92087e67200b None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:50:55.581912 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b156ff-3d5f-43aa-a2cd-92087e67200b None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:50:55.583233 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1211/2435] 10.222.0.15 () {54 vars in 921 bytes} [Wed Aug 24 05:50:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:50:55.589335 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abbd548-6425-4e76-a437-d88f4ffd0e2c None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:50:55.591934 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abbd548-6425-4e76-a437-d88f4ffd0e2c None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:50:55.591934 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abbd548-6425-4e76-a437-d88f4ffd0e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:50:55.610083 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-8abbd548-6425-4e76-a437-d88f4ffd0e2c None None] MFA Rules not processed for user `fbecffb800584c5786813accd061421b`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:50:55.611303 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-8abbd548-6425-4e76-a437-d88f4ffd0e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:50:55.621235 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1225/2436] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Aug 24 05:50:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:50:55.633233 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b156ff-3d5f-43aa-a2cd-92087e67200b None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:50:55.645463 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b156ff-3d5f-43aa-a2cd-92087e67200b None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:50:55.650101 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c740f-126d-40be-8006-d5eb46897869 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:50:55.650709 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c740f-126d-40be-8006-d5eb46897869 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:50:55.650993 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c740f-126d-40be-8006-d5eb46897869 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:50:55.651282 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c740f-126d-40be-8006-d5eb46897869 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:50:55.661554 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977c740f-126d-40be-8006-d5eb46897869 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:50:55.666423 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977c740f-126d-40be-8006-d5eb46897869 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:50:55.675230 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1212/2437] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:50:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:50:55.737951 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abbd548-6425-4e76-a437-d88f4ffd0e2c None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:50:55.739341 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-8abbd548-6425-4e76-a437-d88f4ffd0e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:50:55.808854 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abbd548-6425-4e76-a437-d88f4ffd0e2c None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:50:55.815493 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1dbc4f-dee4-46b2-86b2-5bba4605ff7f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6778b7c5ad19484b877494f143050405', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:50:55.816113 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1dbc4f-dee4-46b2-86b2-5bba4605ff7f service placement] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:50:55.816421 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1dbc4f-dee4-46b2-86b2-5bba4605ff7f service placement] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:50:55.816720 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1dbc4f-dee4-46b2-86b2-5bba4605ff7f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:50:55.819968 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-3a1dbc4f-dee4-46b2-86b2-5bba4605ff7f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:50:55.832761 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1dbc4f-dee4-46b2-86b2-5bba4605ff7f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:50:55.835464 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1dbc4f-dee4-46b2-86b2-5bba4605ff7f service placement] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:50:55.852236 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1226/2438] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:50:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:50:59.728738 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a96cb-12ff-474e-b8a3-c817d6e214e6 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:50:59.731123 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a96cb-12ff-474e-b8a3-c817d6e214e6 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:50:59.731123 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a96cb-12ff-474e-b8a3-c817d6e214e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:50:59.754520 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-cb1a96cb-12ff-474e-b8a3-c817d6e214e6 None None] MFA Rules not processed for user `604380c191c8461bbb937b8d951238bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:50:59.757184 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-cb1a96cb-12ff-474e-b8a3-c817d6e214e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:50:59.767220 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1213/2439] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Aug 24 05:50:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:50:59.783140 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1dbc4f-dee4-46b2-86b2-5bba4605ff7f service placement] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:50:59.798356 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1dbc4f-dee4-46b2-86b2-5bba4605ff7f service placement] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:50:59.804661 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd7aba-4e00-4547-9856-14a45c068e12 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb2b6cae4a004539a95fc7dd558db255', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:50:59.805376 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd7aba-4e00-4547-9856-14a45c068e12 service swift] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:50:59.805564 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd7aba-4e00-4547-9856-14a45c068e12 service swift] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:50:59.805844 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd7aba-4e00-4547-9856-14a45c068e12 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:50:59.814973 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fd7aba-4e00-4547-9856-14a45c068e12 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:50:59.816880 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fd7aba-4e00-4547-9856-14a45c068e12 service swift] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:50:59.826871 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1227/2440] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Aug 24 05:50:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 05:51:09.745326 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=38 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:51:09.823081 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=56 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:51:25.251066 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd7786a-ba2c-448a-b6c2-876182d247a2 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:51:25.251643 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd7786a-ba2c-448a-b6c2-876182d247a2 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:51:25.251643 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd7786a-ba2c-448a-b6c2-876182d247a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:51:25.271135 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-8dd7786a-ba2c-448a-b6c2-876182d247a2 None None] MFA Rules not processed for user `926115c2e54741e6a3aca02437309b8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:51:25.272714 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-8dd7786a-ba2c-448a-b6c2-876182d247a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:51:25.279736 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1214/2441] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:51:25 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:51:26.098285 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd7aba-4e00-4547-9856-14a45c068e12 service swift] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:51:26.099660 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-72fd7aba-4e00-4547-9856-14a45c068e12 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:51:26.114730 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd7aba-4e00-4547-9856-14a45c068e12 service swift] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:51:26.120673 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df62e41-ea39-4861-997d-dee60d12d670 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:51:26.121274 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df62e41-ea39-4861-997d-dee60d12d670 service nova] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:51:26.121553 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df62e41-ea39-4861-997d-dee60d12d670 service nova] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:51:26.121826 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df62e41-ea39-4861-997d-dee60d12d670 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:51:26.123526 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-0df62e41-ea39-4861-997d-dee60d12d670 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:51:26.187176 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df62e41-ea39-4861-997d-dee60d12d670 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:51:26.188871 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df62e41-ea39-4861-997d-dee60d12d670 service nova] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:51:26.199175 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1228/2442] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:51:26 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:51:33.024677 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd7786a-ba2c-448a-b6c2-876182d247a2 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:51:33.039948 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd7786a-ba2c-448a-b6c2-876182d247a2 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:51:33.048436 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d1d31-7463-4d68-a5d0-170fa1d35e22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:51:33.048436 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d1d31-7463-4d68-a5d0-170fa1d35e22 service nova] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:51:33.048436 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d1d31-7463-4d68-a5d0-170fa1d35e22 service nova] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:51:33.048436 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d1d31-7463-4d68-a5d0-170fa1d35e22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:51:33.050337 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-730d1d31-7463-4d68-a5d0-170fa1d35e22 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:51:33.064375 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d1d31-7463-4d68-a5d0-170fa1d35e22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:51:33.065997 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d1d31-7463-4d68-a5d0-170fa1d35e22 service nova] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:51:33.078904 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1215/2443] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:51:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:51:34.131173 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5351a1e-f8d3-48a4-83c5-d0cd3666194d None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:51:34.131937 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5351a1e-f8d3-48a4-83c5-d0cd3666194d None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:51:34.131937 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5351a1e-f8d3-48a4-83c5-d0cd3666194d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:51:34.152828 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-f5351a1e-f8d3-48a4-83c5-d0cd3666194d None None] MFA Rules not processed for user `604380c191c8461bbb937b8d951238bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:51:34.154070 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-f5351a1e-f8d3-48a4-83c5-d0cd3666194d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:51:34.167017 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1229/2444] 10.222.0.15 () {58 vars in 967 bytes} [Wed Aug 24 05:51:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:51:34.240723 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d1d31-7463-4d68-a5d0-170fa1d35e22 service nova] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:51:34.240723 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d1d31-7463-4d68-a5d0-170fa1d35e22 service nova] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:51:34.240723 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56bfe4e-53fd-41f5-8c07-3c3b876cdda8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb2b6cae4a004539a95fc7dd558db255', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:51:34.240723 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56bfe4e-53fd-41f5-8c07-3c3b876cdda8 service swift] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:51:34.243951 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56bfe4e-53fd-41f5-8c07-3c3b876cdda8 service swift] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:51:34.243951 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56bfe4e-53fd-41f5-8c07-3c3b876cdda8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:51:34.243951 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56bfe4e-53fd-41f5-8c07-3c3b876cdda8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:51:34.243951 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56bfe4e-53fd-41f5-8c07-3c3b876cdda8 service swift] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:51:34.243951 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1216/2445] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Aug 24 05:51:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 05:51:44.146635 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=31 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:51:44.243939 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=15 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:52:00.647963 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6132fd-6fae-422b-bea9-1128c0884897 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:52:00.648519 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6132fd-6fae-422b-bea9-1128c0884897 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:52:00.648631 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6132fd-6fae-422b-bea9-1128c0884897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:52:00.670763 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-3e6132fd-6fae-422b-bea9-1128c0884897 None None] MFA Rules not processed for user `d3e6e20d5cd44444b43b1e7abb240f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:52:00.675032 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-3e6132fd-6fae-422b-bea9-1128c0884897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:52:00.736659 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1230/2446] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:52:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:52:10.729646 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=25 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:52:37.428007 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56bfe4e-53fd-41f5-8c07-3c3b876cdda8 service swift] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:52:37.429915 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-e56bfe4e-53fd-41f5-8c07-3c3b876cdda8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:52:37.513018 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56bfe4e-53fd-41f5-8c07-3c3b876cdda8 service swift] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:52:37.518020 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b880d5-e4ce-48b2-88ba-9c2c0fbe95ba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6778b7c5ad19484b877494f143050405', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:52:37.518617 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b880d5-e4ce-48b2-88ba-9c2c0fbe95ba service placement] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:52:37.519421 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b880d5-e4ce-48b2-88ba-9c2c0fbe95ba service placement] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:52:37.519696 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b880d5-e4ce-48b2-88ba-9c2c0fbe95ba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:52:37.521578 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-25b880d5-e4ce-48b2-88ba-9c2c0fbe95ba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:52:37.533664 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b880d5-e4ce-48b2-88ba-9c2c0fbe95ba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:52:37.535972 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b880d5-e4ce-48b2-88ba-9c2c0fbe95ba service placement] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:52:37.546194 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1217/2447] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:52:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:52:47.542008 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=37 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:02.580426 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b3d65e-5e95-4c91-88d9-db51f441bfa2 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:02.580885 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b3d65e-5e95-4c91-88d9-db51f441bfa2 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:02.580993 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b3d65e-5e95-4c91-88d9-db51f441bfa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:02.602827 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-68b3d65e-5e95-4c91-88d9-db51f441bfa2 None None] MFA Rules not processed for user `b92925514b544fbe9e34326f9c48cb24`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:53:02.604046 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-68b3d65e-5e95-4c91-88d9-db51f441bfa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:02.663664 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1231/2448] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:53:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:53:02.982597 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b880d5-e4ce-48b2-88ba-9c2c0fbe95ba service placement] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:03.002232 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b880d5-e4ce-48b2-88ba-9c2c0fbe95ba service placement] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:03.005886 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4632e961-9843-465e-b9d4-ac1e4042973e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:03.008764 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632e961-9843-465e-b9d4-ac1e4042973e service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:03.008764 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632e961-9843-465e-b9d4-ac1e4042973e service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:03.008764 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632e961-9843-465e-b9d4-ac1e4042973e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:03.009108 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-4632e961-9843-465e-b9d4-ac1e4042973e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:03.030873 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4632e961-9843-465e-b9d4-ac1e4042973e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:03.033532 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4632e961-9843-465e-b9d4-ac1e4042973e service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:03.041125 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1218/2449] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:53:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:53:12.654414 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=25 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:13.037464 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=15 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:15.759241 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b3d65e-5e95-4c91-88d9-db51f441bfa2 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:15.801292 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b3d65e-5e95-4c91-88d9-db51f441bfa2 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:15.807582 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018f8fb0-6f75-43fa-84bd-9e37cf221c63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:15.808457 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f8fb0-6f75-43fa-84bd-9e37cf221c63 service nova] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:15.808838 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f8fb0-6f75-43fa-84bd-9e37cf221c63 service nova] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:15.809180 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f8fb0-6f75-43fa-84bd-9e37cf221c63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:15.811071 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-018f8fb0-6f75-43fa-84bd-9e37cf221c63 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:15.922883 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018f8fb0-6f75-43fa-84bd-9e37cf221c63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:15.922883 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018f8fb0-6f75-43fa-84bd-9e37cf221c63 service nova] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:15.924263 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1232/2450] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:53:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:53:21.313214 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f8af5-3ac8-49dc-a433-1e49f9d47bd5 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:21.313777 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f8af5-3ac8-49dc-a433-1e49f9d47bd5 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:21.313934 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f8af5-3ac8-49dc-a433-1e49f9d47bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:21.336355 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-1e0f8af5-3ac8-49dc-a433-1e49f9d47bd5 None None] MFA Rules not processed for user `c7ef3bf8dfd342f5a2fef02bf597af90`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:53:21.337659 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-1e0f8af5-3ac8-49dc-a433-1e49f9d47bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:21.352064 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1219/2451] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:53:21 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:53:25.920026 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=35 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:29.049782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef75f6-31e1-457c-b1a4-aedc4545a563 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:29.050391 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef75f6-31e1-457c-b1a4-aedc4545a563 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:29.050496 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef75f6-31e1-457c-b1a4-aedc4545a563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:29.085700 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-6aef75f6-31e1-457c-b1a4-aedc4545a563 None None] MFA Rules not processed for user `604380c191c8461bbb937b8d951238bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:53:29.087786 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-6aef75f6-31e1-457c-b1a4-aedc4545a563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:29.096466 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1233/2452] 10.222.0.15 () {58 vars in 967 bytes} [Wed Aug 24 05:53:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:53:29.293037 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f8af5-3ac8-49dc-a433-1e49f9d47bd5 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:29.306552 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f8af5-3ac8-49dc-a433-1e49f9d47bd5 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:29.312344 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e0e81-a8a6-4da4-a4c3-39ca92815378 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb2b6cae4a004539a95fc7dd558db255', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:29.313351 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e0e81-a8a6-4da4-a4c3-39ca92815378 service swift] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:29.313890 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e0e81-a8a6-4da4-a4c3-39ca92815378 service swift] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:29.314417 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e0e81-a8a6-4da4-a4c3-39ca92815378 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:29.336264 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01e0e81-a8a6-4da4-a4c3-39ca92815378 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:29.338998 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01e0e81-a8a6-4da4-a4c3-39ca92815378 service swift] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:29.354257 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1220/2453] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Aug 24 05:53:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 05:53:30.773212 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aef75f6-31e1-457c-b1a4-aedc4545a563 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:30.783373 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aef75f6-31e1-457c-b1a4-aedc4545a563 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:30.787298 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690cc991-5cc0-46ca-8233-af71b17ca02b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:30.787907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690cc991-5cc0-46ca-8233-af71b17ca02b service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:30.788194 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690cc991-5cc0-46ca-8233-af71b17ca02b service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:30.788506 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690cc991-5cc0-46ca-8233-af71b17ca02b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:30.790303 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-690cc991-5cc0-46ca-8233-af71b17ca02b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:30.927811 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690cc991-5cc0-46ca-8233-af71b17ca02b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:30.931070 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690cc991-5cc0-46ca-8233-af71b17ca02b service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:30.951551 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e0e81-a8a6-4da4-a4c3-39ca92815378 service swift] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:30.961576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1234/2454] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:53:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:53:30.967168 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e0e81-a8a6-4da4-a4c3-39ca92815378 service swift] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:30.973053 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f57c79-ee9c-4cd4-80df-d15d054c581d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:30.975411 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f57c79-ee9c-4cd4-80df-d15d054c581d service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:30.975411 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f57c79-ee9c-4cd4-80df-d15d054c581d service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:30.975411 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f57c79-ee9c-4cd4-80df-d15d054c581d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:30.978246 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-98f57c79-ee9c-4cd4-80df-d15d054c581d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:30.994863 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f57c79-ee9c-4cd4-80df-d15d054c581d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:30.996832 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f57c79-ee9c-4cd4-80df-d15d054c581d service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:31.014210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1221/2455] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:53:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:53:35.843070 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690cc991-5cc0-46ca-8233-af71b17ca02b service neutron] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:35.861330 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690cc991-5cc0-46ca-8233-af71b17ca02b service neutron] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:35.868742 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8922e1bd-7f29-437d-ae18-98b43ad8036f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6778b7c5ad19484b877494f143050405', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:35.869628 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922e1bd-7f29-437d-ae18-98b43ad8036f service placement] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:35.870138 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922e1bd-7f29-437d-ae18-98b43ad8036f service placement] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:35.870575 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922e1bd-7f29-437d-ae18-98b43ad8036f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:35.872819 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-8922e1bd-7f29-437d-ae18-98b43ad8036f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:35.895220 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8922e1bd-7f29-437d-ae18-98b43ad8036f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:35.897006 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8922e1bd-7f29-437d-ae18-98b43ad8036f service placement] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:35.912734 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1235/2456] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:53:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:53:36.017925 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f57c79-ee9c-4cd4-80df-d15d054c581d service neutron] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:36.019418 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-98f57c79-ee9c-4cd4-80df-d15d054c581d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:36.032509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f57c79-ee9c-4cd4-80df-d15d054c581d service neutron] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:36.045520 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57706b4b-ac35-488b-bb36-d24765645384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:36.048704 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57706b4b-ac35-488b-bb36-d24765645384 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:36.049878 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57706b4b-ac35-488b-bb36-d24765645384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:36.052065 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57706b4b-ac35-488b-bb36-d24765645384 admin admin] PATH_INFO: `/v3/users/926115c2e54741e6a3aca02437309b8f` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:36.057842 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57706b4b-ac35-488b-bb36-d24765645384 admin admin] RBAC: Authorizing `identity:delete_user(user_id=926115c2e54741e6a3aca02437309b8f)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:36.057842 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57706b4b-ac35-488b-bb36-d24765645384 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:36.368296 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-57706b4b-ac35-488b-bb36-d24765645384 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '926115c2e54741e6a3aca02437309b8f'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:36.375607 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-57706b4b-ac35-488b-bb36-d24765645384 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '926115c2e54741e6a3aca02437309b8f'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:36.376320 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-57706b4b-ac35-488b-bb36-d24765645384 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '926115c2e54741e6a3aca02437309b8f'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:36.440800 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-57706b4b-ac35-488b-bb36-d24765645384 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '926115c2e54741e6a3aca02437309b8f'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:36.453922 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-57706b4b-ac35-488b-bb36-d24765645384 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '926115c2e54741e6a3aca02437309b8f'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:36.461664 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1222/2457] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:53:36 2022] DELETE /identity/v3/users/926115c2e54741e6a3aca02437309b8f => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:53:36.811076 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8922e1bd-7f29-437d-ae18-98b43ad8036f service placement] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:36.823798 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8922e1bd-7f29-437d-ae18-98b43ad8036f service placement] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:36.899077 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2813ccd-25a4-4c41-9fc3-32d5c8b87622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:36.900005 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2813ccd-25a4-4c41-9fc3-32d5c8b87622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:36.900439 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2813ccd-25a4-4c41-9fc3-32d5c8b87622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:36.900802 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2813ccd-25a4-4c41-9fc3-32d5c8b87622 admin admin] PATH_INFO: `/v3/projects/39a99050c88d48db851f437f68b65a58` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:36.902231 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2813ccd-25a4-4c41-9fc3-32d5c8b87622 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39a99050c88d48db851f437f68b65a58)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:36.904269 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2813ccd-25a4-4c41-9fc3-32d5c8b87622 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:37.019193 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:37.026974 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-a2813ccd-25a4-4c41-9fc3-32d5c8b87622 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39a99050c88d48db851f437f68b65a58', 'request_id': 'req-a2813ccd-25a4-4c41-9fc3-32d5c8b87622'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:37.033476 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-a2813ccd-25a4-4c41-9fc3-32d5c8b87622 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39a99050c88d48db851f437f68b65a58', 'request_id': 'req-a2813ccd-25a4-4c41-9fc3-32d5c8b87622'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:37.157082 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-a2813ccd-25a4-4c41-9fc3-32d5c8b87622 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39a99050c88d48db851f437f68b65a58', 'request_id': 'req-a2813ccd-25a4-4c41-9fc3-32d5c8b87622'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:37.167726 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-a2813ccd-25a4-4c41-9fc3-32d5c8b87622 admin admin] The token cache is being invalidate because project 39a99050c88d48db851f437f68b65a58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:53:37.168009 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-a2813ccd-25a4-4c41-9fc3-32d5c8b87622 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:37.168670 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-a2813ccd-25a4-4c41-9fc3-32d5c8b87622 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:37.170192 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1236/2458] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:53:36 2022] DELETE /identity/v3/projects/39a99050c88d48db851f437f68b65a58 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:53:37.175672 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62840f-ddcd-4b63-ae05-4cada9266a4c None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:37.176153 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62840f-ddcd-4b63-ae05-4cada9266a4c None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:37.176271 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62840f-ddcd-4b63-ae05-4cada9266a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:37.242571 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-1a62840f-ddcd-4b63-ae05-4cada9266a4c None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:53:37.243840 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-1a62840f-ddcd-4b63-ae05-4cada9266a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:37.340846 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1223/2459] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:53:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:53:37.341124 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd2338b-c7bc-4e94-9a5b-cadad930dc0c None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:37.341124 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd2338b-c7bc-4e94-9a5b-cadad930dc0c None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:37.341124 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd2338b-c7bc-4e94-9a5b-cadad930dc0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:37.366270 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-cfd2338b-c7bc-4e94-9a5b-cadad930dc0c None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:53:37.367845 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-cfd2338b-c7bc-4e94-9a5b-cadad930dc0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:37.378970 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1237/2460] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:53:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:53:37.383977 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62840f-ddcd-4b63-ae05-4cada9266a4c None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:37.395707 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62840f-ddcd-4b63-ae05-4cada9266a4c None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:37.402218 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b9f62c-80d4-4684-9f3e-980dafe05272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:37.403126 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9f62c-80d4-4684-9f3e-980dafe05272 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:37.403504 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9f62c-80d4-4684-9f3e-980dafe05272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:37.403795 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9f62c-80d4-4684-9f3e-980dafe05272 admin admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:37.404521 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b9f62c-80d4-4684-9f3e-980dafe05272 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:53:37.404836 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b9f62c-80d4-4684-9f3e-980dafe05272 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:37.407212 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b9f62c-80d4-4684-9f3e-980dafe05272 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:37.411336 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:37.422864 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1224/2461] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:53:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:53:37.427936 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd2338b-c7bc-4e94-9a5b-cadad930dc0c None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:37.439912 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd2338b-c7bc-4e94-9a5b-cadad930dc0c None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:37.445951 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83022fae-bd0b-4123-ae1f-ccf330e83fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:37.446661 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83022fae-bd0b-4123-ae1f-ccf330e83fb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:37.447008 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83022fae-bd0b-4123-ae1f-ccf330e83fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:37.447346 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83022fae-bd0b-4123-ae1f-ccf330e83fb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:37.448057 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83022fae-bd0b-4123-ae1f-ccf330e83fb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:37.450225 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83022fae-bd0b-4123-ae1f-ccf330e83fb2 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:37.599511 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1238/2462] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:53:37 2022] POST /identity/v3/projects => generated 375 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:53:37.603557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b9f62c-80d4-4684-9f3e-980dafe05272 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:37.614945 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b9f62c-80d4-4684-9f3e-980dafe05272 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:37.677054 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e7bba-d795-4522-8217-c5a5e29798ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:37.677878 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e7bba-d795-4522-8217-c5a5e29798ed admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:37.678166 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e7bba-d795-4522-8217-c5a5e29798ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:37.678446 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e7bba-d795-4522-8217-c5a5e29798ed admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:37.679096 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31e7bba-d795-4522-8217-c5a5e29798ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:37.681092 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31e7bba-d795-4522-8217-c5a5e29798ed admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:37.861324 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83022fae-bd0b-4123-ae1f-ccf330e83fb2 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:37.861559 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1225/2463] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:53:37 2022] POST /identity/v3/users => generated 288 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:53:37.874843 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83022fae-bd0b-4123-ae1f-ccf330e83fb2 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:37.892491 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea720d4-b5ec-42c0-aea8-4235994a4084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:37.895911 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea720d4-b5ec-42c0-aea8-4235994a4084 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:37.895911 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea720d4-b5ec-42c0-aea8-4235994a4084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:37.896275 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea720d4-b5ec-42c0-aea8-4235994a4084 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:37.898192 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea720d4-b5ec-42c0-aea8-4235994a4084 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:37.901237 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea720d4-b5ec-42c0-aea8-4235994a4084 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:37.941304 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1239/2464] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:53:37 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:53:37.944982 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e7bba-d795-4522-8217-c5a5e29798ed admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:37.963579 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e7bba-d795-4522-8217-c5a5e29798ed admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:37.977358 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558854d-3454-4c6b-a547-c360d7c69d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:37.980511 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558854d-3454-4c6b-a547-c360d7c69d70 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:37.980511 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558854d-3454-4c6b-a547-c360d7c69d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:37.980511 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558854d-3454-4c6b-a547-c360d7c69d70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:37.980511 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e558854d-3454-4c6b-a547-c360d7c69d70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:37.981550 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e558854d-3454-4c6b-a547-c360d7c69d70 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:38.003965 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1226/2465] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:53:37 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:53:38.014081 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea720d4-b5ec-42c0-aea8-4235994a4084 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:38.031517 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea720d4-b5ec-42c0-aea8-4235994a4084 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:38.039547 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d50d5e-4b8a-4bf4-8f6f-47eb36e39492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:38.040482 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d50d5e-4b8a-4bf4-8f6f-47eb36e39492 admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:38.040767 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d50d5e-4b8a-4bf4-8f6f-47eb36e39492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:38.041042 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d50d5e-4b8a-4bf4-8f6f-47eb36e39492 admin admin] PATH_INFO: `/v3/projects/dd4a461b2a2645908c8229d77e13c743/users/340c99a3e19343aab956056060a23896/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:38.060326 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d50d5e-4b8a-4bf4-8f6f-47eb36e39492 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd4a461b2a2645908c8229d77e13c743, user_id=340c99a3e19343aab956056060a23896, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:38.063034 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d50d5e-4b8a-4bf4-8f6f-47eb36e39492 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:38.183886 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1240/2466] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:53:38 2022] PUT /identity/v3/projects/dd4a461b2a2645908c8229d77e13c743/users/340c99a3e19343aab956056060a23896/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:53:38.193775 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d532c6a6-138b-4085-87b2-36f64558fda8 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:38.196689 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d532c6a6-138b-4085-87b2-36f64558fda8 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:38.197183 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d532c6a6-138b-4085-87b2-36f64558fda8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:38.221000 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-d532c6a6-138b-4085-87b2-36f64558fda8 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:53:38.222698 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-d532c6a6-138b-4085-87b2-36f64558fda8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:38.327913 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1227/2467] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:53:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:53:38.340120 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d50d5e-4b8a-4bf4-8f6f-47eb36e39492 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:38.341955 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-04d50d5e-4b8a-4bf4-8f6f-47eb36e39492 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:38.447016 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d50d5e-4b8a-4bf4-8f6f-47eb36e39492 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:38.450954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f75e57-7454-4362-85fa-5b0602d75e59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:38.451644 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f75e57-7454-4362-85fa-5b0602d75e59 service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:38.452005 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f75e57-7454-4362-85fa-5b0602d75e59 service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:38.452388 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f75e57-7454-4362-85fa-5b0602d75e59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:38.463915 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f75e57-7454-4362-85fa-5b0602d75e59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:38.465503 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f75e57-7454-4362-85fa-5b0602d75e59 service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:38.477491 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1241/2468] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:53:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:53:40.334870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d532c6a6-138b-4085-87b2-36f64558fda8 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:40.336419 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-d532c6a6-138b-4085-87b2-36f64558fda8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:40.436532 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d532c6a6-138b-4085-87b2-36f64558fda8 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:40.442911 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e9e877-12b2-446d-aed1-144c59550de9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6778b7c5ad19484b877494f143050405', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:40.443830 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e9e877-12b2-446d-aed1-144c59550de9 service placement] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:40.444280 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e9e877-12b2-446d-aed1-144c59550de9 service placement] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:40.444736 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e9e877-12b2-446d-aed1-144c59550de9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:40.447147 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-38e9e877-12b2-446d-aed1-144c59550de9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:40.460774 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e9e877-12b2-446d-aed1-144c59550de9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:40.462711 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e9e877-12b2-446d-aed1-144c59550de9 service placement] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:40.483912 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f75e57-7454-4362-85fa-5b0602d75e59 service neutron] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:40.486227 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-e9f75e57-7454-4362-85fa-5b0602d75e59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:40.506688 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1228/2469] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:53:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:53:40.549431 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f75e57-7454-4362-85fa-5b0602d75e59 service neutron] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:40.594048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd0054a-5759-4955-a46c-29ec3fbf83b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:40.595550 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd0054a-5759-4955-a46c-29ec3fbf83b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:40.595843 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd0054a-5759-4955-a46c-29ec3fbf83b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:40.596131 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd0054a-5759-4955-a46c-29ec3fbf83b1 admin admin] PATH_INFO: `/v3/users/358428458beb4ad6820c6d40d39390e4` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:40.597647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd0054a-5759-4955-a46c-29ec3fbf83b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=358428458beb4ad6820c6d40d39390e4)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:40.600665 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd0054a-5759-4955-a46c-29ec3fbf83b1 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:40.895013 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-fcd0054a-5759-4955-a46c-29ec3fbf83b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '358428458beb4ad6820c6d40d39390e4'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:41.001832 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-fcd0054a-5759-4955-a46c-29ec3fbf83b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '358428458beb4ad6820c6d40d39390e4'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:41.012740 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-fcd0054a-5759-4955-a46c-29ec3fbf83b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '358428458beb4ad6820c6d40d39390e4'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:41.016622 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=84,keystone:DELETE=12,keystone:INSERT=4 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:41.029590 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-fcd0054a-5759-4955-a46c-29ec3fbf83b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '358428458beb4ad6820c6d40d39390e4'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:41.030629 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-fcd0054a-5759-4955-a46c-29ec3fbf83b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '358428458beb4ad6820c6d40d39390e4'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:41.042328 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1242/2470] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:53:40 2022] DELETE /identity/v3/users/358428458beb4ad6820c6d40d39390e4 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:53:41.428421 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e9e877-12b2-446d-aed1-144c59550de9 service placement] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:41.435221 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:41.442559 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e9e877-12b2-446d-aed1-144c59550de9 service placement] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:41.518190 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb407155-e457-4f2c-8e4a-747d2a973ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:41.518865 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb407155-e457-4f2c-8e4a-747d2a973ba9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:41.519159 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb407155-e457-4f2c-8e4a-747d2a973ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:41.519440 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb407155-e457-4f2c-8e4a-747d2a973ba9 admin admin] PATH_INFO: `/v3/projects/de8b104cdda04729a55a5af9d4503c81` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:41.521027 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb407155-e457-4f2c-8e4a-747d2a973ba9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de8b104cdda04729a55a5af9d4503c81)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:41.523013 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb407155-e457-4f2c-8e4a-747d2a973ba9 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:41.646614 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-fb407155-e457-4f2c-8e4a-747d2a973ba9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de8b104cdda04729a55a5af9d4503c81', 'request_id': 'req-fb407155-e457-4f2c-8e4a-747d2a973ba9'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:41.654623 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-fb407155-e457-4f2c-8e4a-747d2a973ba9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de8b104cdda04729a55a5af9d4503c81', 'request_id': 'req-fb407155-e457-4f2c-8e4a-747d2a973ba9'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:41.732317 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-fb407155-e457-4f2c-8e4a-747d2a973ba9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de8b104cdda04729a55a5af9d4503c81', 'request_id': 'req-fb407155-e457-4f2c-8e4a-747d2a973ba9'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:41.746865 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-fb407155-e457-4f2c-8e4a-747d2a973ba9 admin admin] The token cache is being invalidate because project de8b104cdda04729a55a5af9d4503c81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:53:41.747460 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-fb407155-e457-4f2c-8e4a-747d2a973ba9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:41.748276 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-fb407155-e457-4f2c-8e4a-747d2a973ba9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:41.750143 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1229/2471] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:53:41 2022] DELETE /identity/v3/projects/de8b104cdda04729a55a5af9d4503c81 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:53:41.754318 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd0054a-5759-4955-a46c-29ec3fbf83b1 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:41.756813 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-fcd0054a-5759-4955-a46c-29ec3fbf83b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:41.771127 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd0054a-5759-4955-a46c-29ec3fbf83b1 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:41.870449 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f016b2-ff13-4599-bd9d-c86b17116749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:41.871507 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f016b2-ff13-4599-bd9d-c86b17116749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:41.872177 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f016b2-ff13-4599-bd9d-c86b17116749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:41.872858 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f016b2-ff13-4599-bd9d-c86b17116749 admin admin] PATH_INFO: `/v3/users/dcc73f1b94024d4a813ee086de227b38` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:41.874562 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f016b2-ff13-4599-bd9d-c86b17116749 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcc73f1b94024d4a813ee086de227b38)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:41.877040 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f016b2-ff13-4599-bd9d-c86b17116749 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:42.117973 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-42f016b2-ff13-4599-bd9d-c86b17116749 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcc73f1b94024d4a813ee086de227b38'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:42.225107 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-42f016b2-ff13-4599-bd9d-c86b17116749 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcc73f1b94024d4a813ee086de227b38'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:42.251158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-42f016b2-ff13-4599-bd9d-c86b17116749 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcc73f1b94024d4a813ee086de227b38'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:42.276337 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-42f016b2-ff13-4599-bd9d-c86b17116749 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcc73f1b94024d4a813ee086de227b38'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:42.281905 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-42f016b2-ff13-4599-bd9d-c86b17116749 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcc73f1b94024d4a813ee086de227b38'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:42.286919 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1243/2472] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:53:41 2022] DELETE /identity/v3/users/dcc73f1b94024d4a813ee086de227b38 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:53:42.732073 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb407155-e457-4f2c-8e4a-747d2a973ba9 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:42.734738 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-fb407155-e457-4f2c-8e4a-747d2a973ba9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:42.750720 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb407155-e457-4f2c-8e4a-747d2a973ba9 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:42.824539 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c6acd-27a9-4095-b746-f741a368518a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:42.825403 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c6acd-27a9-4095-b746-f741a368518a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:42.825533 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c6acd-27a9-4095-b746-f741a368518a admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:42.825729 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c6acd-27a9-4095-b746-f741a368518a admin admin] PATH_INFO: `/v3/projects/8d0846e5dc424b1b9e3c96604ca3f51f` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:42.826970 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680c6acd-27a9-4095-b746-f741a368518a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d0846e5dc424b1b9e3c96604ca3f51f)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:42.829600 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680c6acd-27a9-4095-b746-f741a368518a admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:42.944793 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:42.960830 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-680c6acd-27a9-4095-b746-f741a368518a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d0846e5dc424b1b9e3c96604ca3f51f', 'request_id': 'req-680c6acd-27a9-4095-b746-f741a368518a'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:42.971869 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-680c6acd-27a9-4095-b746-f741a368518a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d0846e5dc424b1b9e3c96604ca3f51f', 'request_id': 'req-680c6acd-27a9-4095-b746-f741a368518a'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:43.032529 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-680c6acd-27a9-4095-b746-f741a368518a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d0846e5dc424b1b9e3c96604ca3f51f', 'request_id': 'req-680c6acd-27a9-4095-b746-f741a368518a'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:43.056246 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-680c6acd-27a9-4095-b746-f741a368518a admin admin] The token cache is being invalidate because project 8d0846e5dc424b1b9e3c96604ca3f51f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:53:43.056854 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-680c6acd-27a9-4095-b746-f741a368518a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:43.057768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-680c6acd-27a9-4095-b746-f741a368518a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:43.072476 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1230/2473] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:53:42 2022] DELETE /identity/v3/projects/8d0846e5dc424b1b9e3c96604ca3f51f => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:53:45.029036 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce65d3b-7d1e-476e-b7eb-ac717de30c06 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:45.029632 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce65d3b-7d1e-476e-b7eb-ac717de30c06 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:45.029632 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce65d3b-7d1e-476e-b7eb-ac717de30c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:45.059838 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-6ce65d3b-7d1e-476e-b7eb-ac717de30c06 None None] MFA Rules not processed for user `340c99a3e19343aab956056060a23896`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:53:45.061122 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-6ce65d3b-7d1e-476e-b7eb-ac717de30c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:45.113320 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:45.138927 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1244/2474] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:53:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:53:45.155685 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c6acd-27a9-4095-b746-f741a368518a admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:45.156075 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-680c6acd-27a9-4095-b746-f741a368518a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:45.265146 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c6acd-27a9-4095-b746-f741a368518a admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:45.273670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5219a2-de7b-4223-8760-a2ba710f5b57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:45.274298 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5219a2-de7b-4223-8760-a2ba710f5b57 service nova] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:45.274580 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5219a2-de7b-4223-8760-a2ba710f5b57 service nova] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:45.274856 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5219a2-de7b-4223-8760-a2ba710f5b57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:45.283592 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5219a2-de7b-4223-8760-a2ba710f5b57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:45.285162 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5219a2-de7b-4223-8760-a2ba710f5b57 service nova] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:45.299899 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1231/2475] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:53:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:53:47.816569 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba9f96-29eb-48cd-9930-bd9665b1859a None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:47.817410 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba9f96-29eb-48cd-9930-bd9665b1859a None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:47.817697 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba9f96-29eb-48cd-9930-bd9665b1859a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:47.857290 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-89ba9f96-29eb-48cd-9930-bd9665b1859a None None] MFA Rules not processed for user `340c99a3e19343aab956056060a23896`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:53:47.858945 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-89ba9f96-29eb-48cd-9930-bd9665b1859a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:47.873202 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1245/2476] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:53:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:53:55.291302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=50,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:57.845921 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=9 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:22.487843 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f35dd16-ba8d-4448-bd00-d9ca2e8a80c0 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:22.488373 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f35dd16-ba8d-4448-bd00-d9ca2e8a80c0 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:22.488373 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f35dd16-ba8d-4448-bd00-d9ca2e8a80c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:22.639348 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-9f35dd16-ba8d-4448-bd00-d9ca2e8a80c0 None None] MFA Rules not processed for user `d3e6e20d5cd44444b43b1e7abb240f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:54:22.640606 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-9f35dd16-ba8d-4448-bd00-d9ca2e8a80c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:22.696461 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1232/2477] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:54:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:54:24.515091 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba9f96-29eb-48cd-9930-bd9665b1859a None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:24.516659 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-89ba9f96-29eb-48cd-9930-bd9665b1859a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:24.575718 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba9f96-29eb-48cd-9930-bd9665b1859a None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:24.579628 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e40cf-5db5-48f0-8597-0e5bfb56cbfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:24.580238 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e40cf-5db5-48f0-8597-0e5bfb56cbfb service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:24.580658 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e40cf-5db5-48f0-8597-0e5bfb56cbfb service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:24.580837 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e40cf-5db5-48f0-8597-0e5bfb56cbfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:24.583231 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-665e40cf-5db5-48f0-8597-0e5bfb56cbfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:24.597333 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665e40cf-5db5-48f0-8597-0e5bfb56cbfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:24.599003 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665e40cf-5db5-48f0-8597-0e5bfb56cbfb service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:24.610074 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1246/2478] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:54:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:54:25.721515 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f35dd16-ba8d-4448-bd00-d9ca2e8a80c0 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:25.731962 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f35dd16-ba8d-4448-bd00-d9ca2e8a80c0 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:25.735526 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e90de1-9b3d-48c7-9fde-2b6be526f196 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:25.736059 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e90de1-9b3d-48c7-9fde-2b6be526f196 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:25.736335 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e90de1-9b3d-48c7-9fde-2b6be526f196 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:25.736641 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e90de1-9b3d-48c7-9fde-2b6be526f196 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:25.738408 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-32e90de1-9b3d-48c7-9fde-2b6be526f196 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:25.752439 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e90de1-9b3d-48c7-9fde-2b6be526f196 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:25.753881 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e90de1-9b3d-48c7-9fde-2b6be526f196 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:25.765418 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1233/2479] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:54:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:54:34.605284 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=26 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:35.761046 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=35 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:36.255782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e40cf-5db5-48f0-8597-0e5bfb56cbfb service neutron] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:36.266820 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e40cf-5db5-48f0-8597-0e5bfb56cbfb service neutron] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:36.272416 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8c412e-200b-4dcf-b02d-0d9358e05588 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:36.273014 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c412e-200b-4dcf-b02d-0d9358e05588 service nova] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:36.273291 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c412e-200b-4dcf-b02d-0d9358e05588 service nova] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:36.273568 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c412e-200b-4dcf-b02d-0d9358e05588 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:36.275258 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-1e8c412e-200b-4dcf-b02d-0d9358e05588 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:36.299065 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8c412e-200b-4dcf-b02d-0d9358e05588 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:36.300698 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8c412e-200b-4dcf-b02d-0d9358e05588 service nova] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:36.310444 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1247/2480] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:54:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:54:46.306241 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=11 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:55:42.787859 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e90de1-9b3d-48c7-9fde-2b6be526f196 service neutron] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:55:42.789488 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-32e90de1-9b3d-48c7-9fde-2b6be526f196 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:55:42.800557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e90de1-9b3d-48c7-9fde-2b6be526f196 service neutron] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:55:42.804353 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6b8ab-c73b-437c-bea2-db0ebaf2104a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:55:42.805000 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6b8ab-c73b-437c-bea2-db0ebaf2104a service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:55:42.805283 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6b8ab-c73b-437c-bea2-db0ebaf2104a service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:55:42.805557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6b8ab-c73b-437c-bea2-db0ebaf2104a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:55:42.807281 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-24b6b8ab-c73b-437c-bea2-db0ebaf2104a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:55:42.818110 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b6b8ab-c73b-437c-bea2-db0ebaf2104a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:55:42.819536 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b6b8ab-c73b-437c-bea2-db0ebaf2104a service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:55:42.828284 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1234/2481] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:55:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:55:44.796787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8c412e-200b-4dcf-b02d-0d9358e05588 service nova] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:55:44.798258 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-1e8c412e-200b-4dcf-b02d-0d9358e05588 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:55:44.813735 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8c412e-200b-4dcf-b02d-0d9358e05588 service nova] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:55:44.820337 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b658c45-28f8-4ebd-9410-9dcca4afb6ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:55:44.821017 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b658c45-28f8-4ebd-9410-9dcca4afb6ef service nova] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:55:44.821300 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b658c45-28f8-4ebd-9410-9dcca4afb6ef service nova] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:55:44.821585 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b658c45-28f8-4ebd-9410-9dcca4afb6ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:55:44.823345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-2b658c45-28f8-4ebd-9410-9dcca4afb6ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:55:44.836006 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b658c45-28f8-4ebd-9410-9dcca4afb6ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:55:44.837632 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b658c45-28f8-4ebd-9410-9dcca4afb6ef service nova] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:55:44.848934 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1248/2482] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:55:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:55:45.552438 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6b8ab-c73b-437c-bea2-db0ebaf2104a service neutron] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:55:45.554306 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-24b6b8ab-c73b-437c-bea2-db0ebaf2104a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:55:45.633886 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6b8ab-c73b-437c-bea2-db0ebaf2104a service neutron] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:55:45.638308 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a0d906-1be8-4a1b-bcd0-43d6bbb413c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:55:45.638919 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0d906-1be8-4a1b-bcd0-43d6bbb413c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:55:45.639200 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0d906-1be8-4a1b-bcd0-43d6bbb413c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:55:45.639474 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0d906-1be8-4a1b-bcd0-43d6bbb413c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:55:45.641253 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-71a0d906-1be8-4a1b-bcd0-43d6bbb413c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:55:45.701335 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a0d906-1be8-4a1b-bcd0-43d6bbb413c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:55:45.702764 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a0d906-1be8-4a1b-bcd0-43d6bbb413c6 service cinder] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:55:45.712971 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1235/2483] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:55:45 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:55:54.843748 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=6 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:55:55.157238 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc782ea-5235-419b-bd73-9a14a57a8a02 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:55:55.157908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc782ea-5235-419b-bd73-9a14a57a8a02 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:55:55.158444 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc782ea-5235-419b-bd73-9a14a57a8a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:55:55.215067 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-fdc782ea-5235-419b-bd73-9a14a57a8a02 None None] MFA Rules not processed for user `604380c191c8461bbb937b8d951238bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:55:55.216328 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-fdc782ea-5235-419b-bd73-9a14a57a8a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:55:55.284978 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1249/2484] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Aug 24 05:55:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:55:55.451308 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a0d906-1be8-4a1b-bcd0-43d6bbb413c6 service cinder] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:55:55.452755 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-71a0d906-1be8-4a1b-bcd0-43d6bbb413c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:55:55.546797 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:55:55.549713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a0d906-1be8-4a1b-bcd0-43d6bbb413c6 service cinder] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:55:55.554779 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda7d1d-f84a-4dff-93d7-0994859e3b5f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb2b6cae4a004539a95fc7dd558db255', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:55:55.555519 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda7d1d-f84a-4dff-93d7-0994859e3b5f service swift] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:55:55.555960 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda7d1d-f84a-4dff-93d7-0994859e3b5f service swift] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:55:55.556571 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda7d1d-f84a-4dff-93d7-0994859e3b5f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:55:55.565305 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdda7d1d-f84a-4dff-93d7-0994859e3b5f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:55:55.566893 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdda7d1d-f84a-4dff-93d7-0994859e3b5f service swift] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:55:55.576217 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1236/2485] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Aug 24 05:55:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 05:56:01.695502 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992e272-a0a1-4188-87ec-9f1441de1c6b None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:01.696129 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992e272-a0a1-4188-87ec-9f1441de1c6b None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:01.696682 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992e272-a0a1-4188-87ec-9f1441de1c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:01.718789 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-9992e272-a0a1-4188-87ec-9f1441de1c6b None None] MFA Rules not processed for user `b92925514b544fbe9e34326f9c48cb24`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:01.720461 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-9992e272-a0a1-4188-87ec-9f1441de1c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:01.728984 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1250/2486] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:56:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:56:02.901645 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda7d1d-f84a-4dff-93d7-0994859e3b5f service swift] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:02.903309 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-cdda7d1d-f84a-4dff-93d7-0994859e3b5f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:02.994486 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda7d1d-f84a-4dff-93d7-0994859e3b5f service swift] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:03.005150 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee836c3d-4e39-4d44-a168-0e25571ac361 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6778b7c5ad19484b877494f143050405', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:03.006133 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee836c3d-4e39-4d44-a168-0e25571ac361 service placement] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:03.006659 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee836c3d-4e39-4d44-a168-0e25571ac361 service placement] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:03.007055 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee836c3d-4e39-4d44-a168-0e25571ac361 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:03.010528 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-ee836c3d-4e39-4d44-a168-0e25571ac361 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:03.029602 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee836c3d-4e39-4d44-a168-0e25571ac361 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:03.031632 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee836c3d-4e39-4d44-a168-0e25571ac361 service placement] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:03.050415 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1237/2487] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:56:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:56:08.103830 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9992e272-a0a1-4188-87ec-9f1441de1c6b None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:08.134729 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9992e272-a0a1-4188-87ec-9f1441de1c6b None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:08.142564 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af60d667-e078-4a60-82d2-b63e1736be3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:08.143809 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60d667-e078-4a60-82d2-b63e1736be3b service nova] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:08.144273 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60d667-e078-4a60-82d2-b63e1736be3b service nova] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:08.144744 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60d667-e078-4a60-82d2-b63e1736be3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:08.147224 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-af60d667-e078-4a60-82d2-b63e1736be3b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:08.172915 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af60d667-e078-4a60-82d2-b63e1736be3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:08.174820 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af60d667-e078-4a60-82d2-b63e1736be3b service nova] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:08.199904 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1251/2488] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:56:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:56:13.039940 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=36 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:18.189255 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=49 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:18.906950 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27433ecc-d521-454c-9132-92a9c66798f0 None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:18.907509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27433ecc-d521-454c-9132-92a9c66798f0 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:18.907509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27433ecc-d521-454c-9132-92a9c66798f0 None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:18.908933 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1238/2489] 10.222.0.15 () {54 vars in 921 bytes} [Wed Aug 24 05:56:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:56:18.915046 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51139c10-68c0-4372-b0fc-3410ebf79453 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:18.915334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51139c10-68c0-4372-b0fc-3410ebf79453 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:18.915613 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51139c10-68c0-4372-b0fc-3410ebf79453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:18.960255 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-51139c10-68c0-4372-b0fc-3410ebf79453 None None] MFA Rules not processed for user `fbecffb800584c5786813accd061421b`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:18.962311 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-51139c10-68c0-4372-b0fc-3410ebf79453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:18.977043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1252/2490] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Aug 24 05:56:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:56:18.994878 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27433ecc-d521-454c-9132-92a9c66798f0 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:18.996664 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-27433ecc-d521-454c-9132-92a9c66798f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:19.082357 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27433ecc-d521-454c-9132-92a9c66798f0 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:19.085882 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b78a36-adf9-4b79-8ebd-43d820d725c2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9aee9c524a974a248c3bf363444f584a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:19.086624 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b78a36-adf9-4b79-8ebd-43d820d725c2 service glance] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:19.086914 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b78a36-adf9-4b79-8ebd-43d820d725c2 service glance] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:19.087180 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b78a36-adf9-4b79-8ebd-43d820d725c2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:19.098145 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b78a36-adf9-4b79-8ebd-43d820d725c2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:19.099554 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b78a36-adf9-4b79-8ebd-43d820d725c2 service glance] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:19.110181 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1239/2491] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:56:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:56:19.372687 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0189c-d46c-4f32-8989-d61c69e64a2c None None] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:19.373274 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0189c-d46c-4f32-8989-d61c69e64a2c None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:19.373274 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0189c-d46c-4f32-8989-d61c69e64a2c None None] PATH_INFO: `/` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:19.374663 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1253/2492] 10.222.0.15 () {54 vars in 921 bytes} [Wed Aug 24 05:56:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:56:19.379708 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60523ee7-0b63-4dd9-999b-aadf43db4d2a None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:19.380082 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60523ee7-0b63-4dd9-999b-aadf43db4d2a None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:19.380452 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60523ee7-0b63-4dd9-999b-aadf43db4d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:19.423097 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-60523ee7-0b63-4dd9-999b-aadf43db4d2a None None] MFA Rules not processed for user `b1ecd9b302b64ecf81d7e3d3356ddf7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:19.424815 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-60523ee7-0b63-4dd9-999b-aadf43db4d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:19.431839 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1240/2493] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Aug 24 05:56:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:56:19.443708 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0189c-d46c-4f32-8989-d61c69e64a2c None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:19.455479 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0189c-d46c-4f32-8989-d61c69e64a2c None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:19.459266 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d73f90-afe9-4b1c-b874-c862c1102f24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:19.459878 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d73f90-afe9-4b1c-b874-c862c1102f24 service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:19.460161 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d73f90-afe9-4b1c-b874-c862c1102f24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:19.460462 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d73f90-afe9-4b1c-b874-c862c1102f24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:19.468455 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d73f90-afe9-4b1c-b874-c862c1102f24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:19.469888 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d73f90-afe9-4b1c-b874-c862c1102f24 service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:19.478758 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1254/2494] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:56:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:56:19.565946 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e643cc-1517-486a-b16e-e15e51ad123b None None] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:19.566232 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e643cc-1517-486a-b16e-e15e51ad123b None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:19.566505 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e643cc-1517-486a-b16e-e15e51ad123b None None] PATH_INFO: `/` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:19.567812 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1241/2495] 10.222.0.15 () {54 vars in 921 bytes} [Wed Aug 24 05:56:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:56:19.573334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a1028-f7e1-4cca-8257-c3124dd0ded2 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:19.573706 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a1028-f7e1-4cca-8257-c3124dd0ded2 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:19.574059 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a1028-f7e1-4cca-8257-c3124dd0ded2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:19.594289 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-7c6a1028-f7e1-4cca-8257-c3124dd0ded2 None None] MFA Rules not processed for user `fbecffb800584c5786813accd061421b`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:19.595623 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-7c6a1028-f7e1-4cca-8257-c3124dd0ded2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:19.606407 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1255/2496] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Aug 24 05:56:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:56:19.617921 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e643cc-1517-486a-b16e-e15e51ad123b None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:19.630705 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e643cc-1517-486a-b16e-e15e51ad123b None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:19.639700 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a3f7a-0a37-4d6c-8511-b61bb6313b25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:19.640330 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a3f7a-0a37-4d6c-8511-b61bb6313b25 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:19.640636 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a3f7a-0a37-4d6c-8511-b61bb6313b25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:19.640908 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a3f7a-0a37-4d6c-8511-b61bb6313b25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:19.651723 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a3f7a-0a37-4d6c-8511-b61bb6313b25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:19.653171 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a3f7a-0a37-4d6c-8511-b61bb6313b25 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:19.663719 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1242/2497] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:56:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:56:22.449971 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f4c63-3f11-4619-9801-757be527555f None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:22.450429 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f4c63-3f11-4619-9801-757be527555f None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:22.450567 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f4c63-3f11-4619-9801-757be527555f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:22.471759 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-fc1f4c63-3f11-4619-9801-757be527555f None None] MFA Rules not processed for user `604380c191c8461bbb937b8d951238bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:22.473012 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-fc1f4c63-3f11-4619-9801-757be527555f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:22.481191 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1256/2498] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Aug 24 05:56:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:56:22.493355 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a3f7a-0a37-4d6c-8511-b61bb6313b25 service neutron] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:22.504647 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a3f7a-0a37-4d6c-8511-b61bb6313b25 service neutron] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:22.510769 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3202b3ca-05e1-4ea5-9c63-c13e02fd4f03 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb2b6cae4a004539a95fc7dd558db255', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:22.511376 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3202b3ca-05e1-4ea5-9c63-c13e02fd4f03 service swift] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:22.511653 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3202b3ca-05e1-4ea5-9c63-c13e02fd4f03 service swift] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:22.511922 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3202b3ca-05e1-4ea5-9c63-c13e02fd4f03 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:22.521085 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3202b3ca-05e1-4ea5-9c63-c13e02fd4f03 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:22.522652 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3202b3ca-05e1-4ea5-9c63-c13e02fd4f03 service swift] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:22.531893 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1243/2499] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Aug 24 05:56:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 05:56:25.924748 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f4c63-3f11-4619-9801-757be527555f None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:25.938745 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f4c63-3f11-4619-9801-757be527555f None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:25.944253 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0db0ef-7e85-480d-958d-6d5c6ee5bcef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:25.945505 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0db0ef-7e85-480d-958d-6d5c6ee5bcef service nova] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:25.946015 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0db0ef-7e85-480d-958d-6d5c6ee5bcef service nova] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:25.946423 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0db0ef-7e85-480d-958d-6d5c6ee5bcef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:25.948439 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-da0db0ef-7e85-480d-958d-6d5c6ee5bcef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:26.031550 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0db0ef-7e85-480d-958d-6d5c6ee5bcef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:26.033192 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0db0ef-7e85-480d-958d-6d5c6ee5bcef service nova] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:26.052559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1257/2500] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:56:25 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:56:32.529001 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=56 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:36.049722 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=62 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:54.742779 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3202b3ca-05e1-4ea5-9c63-c13e02fd4f03 service swift] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:54.752356 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3202b3ca-05e1-4ea5-9c63-c13e02fd4f03 service swift] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:54.756018 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009b49d0-8ae1-40d0-b0e6-1b58b1c5893d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:54.756992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b49d0-8ae1-40d0-b0e6-1b58b1c5893d service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:54.757307 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b49d0-8ae1-40d0-b0e6-1b58b1c5893d service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:54.757681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b49d0-8ae1-40d0-b0e6-1b58b1c5893d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:54.760094 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-009b49d0-8ae1-40d0-b0e6-1b58b1c5893d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:54.847403 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009b49d0-8ae1-40d0-b0e6-1b58b1c5893d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:54.848867 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009b49d0-8ae1-40d0-b0e6-1b58b1c5893d service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:54.859093 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1244/2501] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:56:54 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:56:58.930582 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6d027-4453-4ae2-890c-eabf944c0da9 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:58.931048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6d027-4453-4ae2-890c-eabf944c0da9 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:58.931147 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6d027-4453-4ae2-890c-eabf944c0da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:58.951234 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-76f6d027-4453-4ae2-890c-eabf944c0da9 None None] MFA Rules not processed for user `604380c191c8461bbb937b8d951238bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:58.952465 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-76f6d027-4453-4ae2-890c-eabf944c0da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:58.960014 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1258/2502] 10.222.0.15 () {58 vars in 967 bytes} [Wed Aug 24 05:56:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:56:58.981941 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009b49d0-8ae1-40d0-b0e6-1b58b1c5893d service neutron] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:58.993691 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009b49d0-8ae1-40d0-b0e6-1b58b1c5893d service neutron] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:58.998633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7aa044-7a89-47a7-9960-670d091c7e88 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb2b6cae4a004539a95fc7dd558db255', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:58.999233 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7aa044-7a89-47a7-9960-670d091c7e88 service swift] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:58.999517 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7aa044-7a89-47a7-9960-670d091c7e88 service swift] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:58.999795 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7aa044-7a89-47a7-9960-670d091c7e88 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:59.009790 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7aa044-7a89-47a7-9960-670d091c7e88 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:59.011363 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7aa044-7a89-47a7-9960-670d091c7e88 service swift] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:59.019507 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1245/2503] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Aug 24 05:56:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 05:57:08.946067 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=3 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:09.016243 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=36 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:15.995974 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6d027-4453-4ae2-890c-eabf944c0da9 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:16.005599 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6d027-4453-4ae2-890c-eabf944c0da9 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:16.009365 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6effe499-406a-4895-a216-ac6d16c0af51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:16.009961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6effe499-406a-4895-a216-ac6d16c0af51 service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:16.010239 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6effe499-406a-4895-a216-ac6d16c0af51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:16.010510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6effe499-406a-4895-a216-ac6d16c0af51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:16.012216 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-6effe499-406a-4895-a216-ac6d16c0af51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:16.077798 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6effe499-406a-4895-a216-ac6d16c0af51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:16.079230 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6effe499-406a-4895-a216-ac6d16c0af51 service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:16.089599 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1259/2504] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:57:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:57:19.594262 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7aa044-7a89-47a7-9960-670d091c7e88 service swift] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:19.595851 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-8d7aa044-7a89-47a7-9960-670d091c7e88 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:19.610671 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7aa044-7a89-47a7-9960-670d091c7e88 service swift] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:19.615962 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9881303-1e3e-4daa-82dc-274677f088d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:19.618021 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9881303-1e3e-4daa-82dc-274677f088d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:19.618021 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9881303-1e3e-4daa-82dc-274677f088d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:19.618021 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9881303-1e3e-4daa-82dc-274677f088d2 admin admin] PATH_INFO: `/v3/users/d3e6e20d5cd44444b43b1e7abb240f8d` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:19.618361 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9881303-1e3e-4daa-82dc-274677f088d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3e6e20d5cd44444b43b1e7abb240f8d)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:19.620269 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9881303-1e3e-4daa-82dc-274677f088d2 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:19.819246 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-d9881303-1e3e-4daa-82dc-274677f088d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3e6e20d5cd44444b43b1e7abb240f8d'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:19.826054 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-d9881303-1e3e-4daa-82dc-274677f088d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3e6e20d5cd44444b43b1e7abb240f8d'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:19.826758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-d9881303-1e3e-4daa-82dc-274677f088d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3e6e20d5cd44444b43b1e7abb240f8d'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:19.879275 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-d9881303-1e3e-4daa-82dc-274677f088d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3e6e20d5cd44444b43b1e7abb240f8d'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:19.893095 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-d9881303-1e3e-4daa-82dc-274677f088d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3e6e20d5cd44444b43b1e7abb240f8d'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:19.899575 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1246/2505] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:57:19 2022] DELETE /identity/v3/users/d3e6e20d5cd44444b43b1e7abb240f8d => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:57:20.150827 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6effe499-406a-4895-a216-ac6d16c0af51 service neutron] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:20.164392 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6effe499-406a-4895-a216-ac6d16c0af51 service neutron] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:20.233832 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39da4058-36a8-45ca-996e-aa71ef06330c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:20.234465 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da4058-36a8-45ca-996e-aa71ef06330c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:20.234747 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da4058-36a8-45ca-996e-aa71ef06330c admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:20.235025 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da4058-36a8-45ca-996e-aa71ef06330c admin admin] PATH_INFO: `/v3/projects/cfbd03315ded40279308fcc7f0dde7cb` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:20.236164 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39da4058-36a8-45ca-996e-aa71ef06330c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfbd03315ded40279308fcc7f0dde7cb)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:20.238154 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39da4058-36a8-45ca-996e-aa71ef06330c admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:20.337348 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-39da4058-36a8-45ca-996e-aa71ef06330c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfbd03315ded40279308fcc7f0dde7cb', 'request_id': 'req-39da4058-36a8-45ca-996e-aa71ef06330c'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:20.343446 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-39da4058-36a8-45ca-996e-aa71ef06330c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfbd03315ded40279308fcc7f0dde7cb', 'request_id': 'req-39da4058-36a8-45ca-996e-aa71ef06330c'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:20.398382 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-39da4058-36a8-45ca-996e-aa71ef06330c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfbd03315ded40279308fcc7f0dde7cb', 'request_id': 'req-39da4058-36a8-45ca-996e-aa71ef06330c'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:20.408872 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-39da4058-36a8-45ca-996e-aa71ef06330c admin admin] The token cache is being invalidate because project cfbd03315ded40279308fcc7f0dde7cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:57:20.410224 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-39da4058-36a8-45ca-996e-aa71ef06330c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:20.410224 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-39da4058-36a8-45ca-996e-aa71ef06330c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:20.411254 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1260/2506] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:57:20 2022] DELETE /identity/v3/projects/cfbd03315ded40279308fcc7f0dde7cb => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:57:20.417259 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29baa7-1d82-4a82-87c6-d8a08a4eac47 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:20.417547 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29baa7-1d82-4a82-87c6-d8a08a4eac47 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:20.417830 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29baa7-1d82-4a82-87c6-d8a08a4eac47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:20.438617 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-3b29baa7-1d82-4a82-87c6-d8a08a4eac47 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:20.439824 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-3b29baa7-1d82-4a82-87c6-d8a08a4eac47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:20.500552 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1247/2507] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:57:20.509041 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e6812-04ed-4fbe-94c9-d2386c927e4a None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:20.509352 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e6812-04ed-4fbe-94c9-d2386c927e4a None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:20.509630 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e6812-04ed-4fbe-94c9-d2386c927e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:20.537530 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-a49e6812-04ed-4fbe-94c9-d2386c927e4a None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:20.538737 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-a49e6812-04ed-4fbe-94c9-d2386c927e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:20.546657 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1261/2508] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:57:20.551170 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b29baa7-1d82-4a82-87c6-d8a08a4eac47 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:20.561655 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b29baa7-1d82-4a82-87c6-d8a08a4eac47 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:20.566714 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1a4227-c3b9-4d20-9501-c134ca40169b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:20.567323 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a4227-c3b9-4d20-9501-c134ca40169b admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:20.567602 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a4227-c3b9-4d20-9501-c134ca40169b admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:20.567880 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a4227-c3b9-4d20-9501-c134ca40169b admin admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:20.568557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1a4227-c3b9-4d20-9501-c134ca40169b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:20.568861 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1a4227-c3b9-4d20-9501-c134ca40169b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:20.570491 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1a4227-c3b9-4d20-9501-c134ca40169b admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:20.583006 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1248/2509] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:57:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:20.587937 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49e6812-04ed-4fbe-94c9-d2386c927e4a None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:20.597161 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49e6812-04ed-4fbe-94c9-d2386c927e4a None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:20.602542 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63629e7-beba-47f5-9972-3db4494e7a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:20.603142 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63629e7-beba-47f5-9972-3db4494e7a6d admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:20.603418 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63629e7-beba-47f5-9972-3db4494e7a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:20.603722 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63629e7-beba-47f5-9972-3db4494e7a6d admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:20.604351 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63629e7-beba-47f5-9972-3db4494e7a6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:20.606287 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63629e7-beba-47f5-9972-3db4494e7a6d admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:20.623981 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:20.716460 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1262/2510] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:57:20 2022] POST /identity/v3/projects => generated 373 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:57:20.720527 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1a4227-c3b9-4d20-9501-c134ca40169b admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:20.730718 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1a4227-c3b9-4d20-9501-c134ca40169b admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:20.790225 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5468cc9-fdde-4910-b96c-1e8c803452bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:20.790837 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5468cc9-fdde-4910-b96c-1e8c803452bd admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:20.791119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5468cc9-fdde-4910-b96c-1e8c803452bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:20.791396 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5468cc9-fdde-4910-b96c-1e8c803452bd admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:20.792019 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5468cc9-fdde-4910-b96c-1e8c803452bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:20.793976 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5468cc9-fdde-4910-b96c-1e8c803452bd admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:20.802092 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:20.878119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1249/2511] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:57:20 2022] POST /identity/v3/users => generated 287 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:57:20.885576 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63629e7-beba-47f5-9972-3db4494e7a6d admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:20.896231 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63629e7-beba-47f5-9972-3db4494e7a6d admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:20.901003 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429e285d-55d0-41dd-b9a6-e1d110dbe954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:20.901611 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e285d-55d0-41dd-b9a6-e1d110dbe954 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:20.901886 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e285d-55d0-41dd-b9a6-e1d110dbe954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:20.902162 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e285d-55d0-41dd-b9a6-e1d110dbe954 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:20.902816 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429e285d-55d0-41dd-b9a6-e1d110dbe954 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:20.904391 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429e285d-55d0-41dd-b9a6-e1d110dbe954 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:20.915307 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1263/2512] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:57:20.919612 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5468cc9-fdde-4910-b96c-1e8c803452bd admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:20.930698 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5468cc9-fdde-4910-b96c-1e8c803452bd admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:20.935808 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28a0474-08a9-4ae7-acf3-083d5e594840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:20.936873 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a0474-08a9-4ae7-acf3-083d5e594840 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:20.938194 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a0474-08a9-4ae7-acf3-083d5e594840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:20.938472 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a0474-08a9-4ae7-acf3-083d5e594840 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:20.939119 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28a0474-08a9-4ae7-acf3-083d5e594840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:20.940771 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28a0474-08a9-4ae7-acf3-083d5e594840 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:20.958401 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1250/2513] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:57:20.967422 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429e285d-55d0-41dd-b9a6-e1d110dbe954 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:20.977783 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429e285d-55d0-41dd-b9a6-e1d110dbe954 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:20.982558 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e79f0-21f6-4251-9f4f-617183b69bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:20.983228 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e79f0-21f6-4251-9f4f-617183b69bf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:20.983511 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e79f0-21f6-4251-9f4f-617183b69bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:20.983791 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e79f0-21f6-4251-9f4f-617183b69bf3 admin admin] PATH_INFO: `/v3/projects/1176334f605d44d099c58dfe8c6776c3/users/d0356dbdbc344b9fbf97f636a8919099/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:21.000966 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70e79f0-21f6-4251-9f4f-617183b69bf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1176334f605d44d099c58dfe8c6776c3, user_id=d0356dbdbc344b9fbf97f636a8919099, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:21.003734 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70e79f0-21f6-4251-9f4f-617183b69bf3 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:21.170475 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1264/2514] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:57:20 2022] PUT /identity/v3/projects/1176334f605d44d099c58dfe8c6776c3/users/d0356dbdbc344b9fbf97f636a8919099/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:57:21.183481 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16cd86d-addd-4fb1-b729-81060e26c064 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:21.183770 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16cd86d-addd-4fb1-b729-81060e26c064 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:21.184050 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16cd86d-addd-4fb1-b729-81060e26c064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:21.214820 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-a16cd86d-addd-4fb1-b729-81060e26c064 None None] MFA Rules not processed for user `d0356dbdbc344b9fbf97f636a8919099`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:21.216040 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-a16cd86d-addd-4fb1-b729-81060e26c064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:21.281474 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1251/2515] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:57:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:57:21.292907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e79f0-21f6-4251-9f4f-617183b69bf3 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:21.294620 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-a70e79f0-21f6-4251-9f4f-617183b69bf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:21.368231 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e79f0-21f6-4251-9f4f-617183b69bf3 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:21.373912 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4196fee-7586-4702-9aa0-c31672b3d1dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:21.374520 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4196fee-7586-4702-9aa0-c31672b3d1dc service nova] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:21.374805 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4196fee-7586-4702-9aa0-c31672b3d1dc service nova] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:21.375082 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4196fee-7586-4702-9aa0-c31672b3d1dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:21.383374 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4196fee-7586-4702-9aa0-c31672b3d1dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:21.385003 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4196fee-7586-4702-9aa0-c31672b3d1dc service nova] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:21.393734 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1265/2516] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:57:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:57:21.634533 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16cd86d-addd-4fb1-b729-81060e26c064 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:21.697750 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16cd86d-addd-4fb1-b729-81060e26c064 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:21.702289 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4a5d7-49d6-4bee-9c75-787e631a5823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:21.702917 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4a5d7-49d6-4bee-9c75-787e631a5823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:21.703198 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4a5d7-49d6-4bee-9c75-787e631a5823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:21.703475 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4a5d7-49d6-4bee-9c75-787e631a5823 admin admin] PATH_INFO: `/v3/users/d0356dbdbc344b9fbf97f636a8919099` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:21.704712 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a4a5d7-49d6-4bee-9c75-787e631a5823 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0356dbdbc344b9fbf97f636a8919099)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:21.706632 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a4a5d7-49d6-4bee-9c75-787e631a5823 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:21.892615 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-37a4a5d7-49d6-4bee-9c75-787e631a5823 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0356dbdbc344b9fbf97f636a8919099'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:21.899655 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-37a4a5d7-49d6-4bee-9c75-787e631a5823 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0356dbdbc344b9fbf97f636a8919099'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:21.900465 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-37a4a5d7-49d6-4bee-9c75-787e631a5823 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0356dbdbc344b9fbf97f636a8919099'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:21.976243 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-37a4a5d7-49d6-4bee-9c75-787e631a5823 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0356dbdbc344b9fbf97f636a8919099'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:21.982057 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:INSERT=4,keystone:SELECT=87,keystone:DELETE=9 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:21.989494 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-37a4a5d7-49d6-4bee-9c75-787e631a5823 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0356dbdbc344b9fbf97f636a8919099'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:21.995457 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1252/2517] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:57:21 2022] DELETE /identity/v3/users/d0356dbdbc344b9fbf97f636a8919099 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:57:22.000223 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7838da-0eff-4e83-822a-d142ac307298 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:22.000676 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7838da-0eff-4e83-822a-d142ac307298 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:22.000957 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7838da-0eff-4e83-822a-d142ac307298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:22.021700 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-ef7838da-0eff-4e83-822a-d142ac307298 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:22.022925 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-ef7838da-0eff-4e83-822a-d142ac307298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:22.082658 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1266/2518] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:57:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:57:22.092597 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4a5d7-49d6-4bee-9c75-787e631a5823 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:22.094075 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-37a4a5d7-49d6-4bee-9c75-787e631a5823 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:22.154506 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4a5d7-49d6-4bee-9c75-787e631a5823 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:22.158308 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd01c49-b728-465f-b474-e63e96259b1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:22.160153 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd01c49-b728-465f-b474-e63e96259b1e service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:22.160153 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd01c49-b728-465f-b474-e63e96259b1e service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:22.160153 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd01c49-b728-465f-b474-e63e96259b1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:22.169318 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd01c49-b728-465f-b474-e63e96259b1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:22.171067 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd01c49-b728-465f-b474-e63e96259b1e service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:22.182998 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1253/2519] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:57:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:57:22.555745 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7838da-0eff-4e83-822a-d142ac307298 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:22.557528 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-ef7838da-0eff-4e83-822a-d142ac307298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:22.577157 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7838da-0eff-4e83-822a-d142ac307298 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:22.583361 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5252a1d5-5b69-44ee-a212-22c286537db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:22.584613 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252a1d5-5b69-44ee-a212-22c286537db1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:22.584986 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252a1d5-5b69-44ee-a212-22c286537db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:22.585236 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252a1d5-5b69-44ee-a212-22c286537db1 admin admin] PATH_INFO: `/v3/projects/1176334f605d44d099c58dfe8c6776c3` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:22.586378 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5252a1d5-5b69-44ee-a212-22c286537db1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1176334f605d44d099c58dfe8c6776c3)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:22.588311 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5252a1d5-5b69-44ee-a212-22c286537db1 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:22.690459 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:22.702811 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-5252a1d5-5b69-44ee-a212-22c286537db1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1176334f605d44d099c58dfe8c6776c3', 'request_id': 'req-5252a1d5-5b69-44ee-a212-22c286537db1'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:22.710356 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-5252a1d5-5b69-44ee-a212-22c286537db1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1176334f605d44d099c58dfe8c6776c3', 'request_id': 'req-5252a1d5-5b69-44ee-a212-22c286537db1'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:22.770348 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-5252a1d5-5b69-44ee-a212-22c286537db1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1176334f605d44d099c58dfe8c6776c3', 'request_id': 'req-5252a1d5-5b69-44ee-a212-22c286537db1'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:22.781785 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-5252a1d5-5b69-44ee-a212-22c286537db1 admin admin] The token cache is being invalidate because project 1176334f605d44d099c58dfe8c6776c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:57:22.782079 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-5252a1d5-5b69-44ee-a212-22c286537db1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:22.782747 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-5252a1d5-5b69-44ee-a212-22c286537db1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:22.784261 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1267/2520] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:57:22 2022] DELETE /identity/v3/projects/1176334f605d44d099c58dfe8c6776c3 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:57:22.790010 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a8d3da-bbc2-4d5f-b19d-d361343c863e None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:22.790300 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a8d3da-bbc2-4d5f-b19d-d361343c863e None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:22.790579 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a8d3da-bbc2-4d5f-b19d-d361343c863e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:22.810644 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-f0a8d3da-bbc2-4d5f-b19d-d361343c863e None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:22.811828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-f0a8d3da-bbc2-4d5f-b19d-d361343c863e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:22.874458 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1254/2521] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:57:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:57:22.883974 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e426e3-e86f-4efd-8d8c-54e0d780dc7f None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:22.884262 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e426e3-e86f-4efd-8d8c-54e0d780dc7f None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:22.884580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e426e3-e86f-4efd-8d8c-54e0d780dc7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:22.904741 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-b7e426e3-e86f-4efd-8d8c-54e0d780dc7f None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:22.905983 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-b7e426e3-e86f-4efd-8d8c-54e0d780dc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:22.916467 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1268/2522] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:57:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:57:22.920298 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a8d3da-bbc2-4d5f-b19d-d361343c863e None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:22.931116 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a8d3da-bbc2-4d5f-b19d-d361343c863e None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:22.935594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159bf7b-edf9-461e-bfdb-8abf79b5aae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:22.936184 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159bf7b-edf9-461e-bfdb-8abf79b5aae2 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:22.936535 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159bf7b-edf9-461e-bfdb-8abf79b5aae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:22.936820 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159bf7b-edf9-461e-bfdb-8abf79b5aae2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:22.937481 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c159bf7b-edf9-461e-bfdb-8abf79b5aae2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:22.937794 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c159bf7b-edf9-461e-bfdb-8abf79b5aae2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:22.939364 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c159bf7b-edf9-461e-bfdb-8abf79b5aae2 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:22.951367 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1255/2523] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:57:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:22.955736 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e426e3-e86f-4efd-8d8c-54e0d780dc7f None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:22.967713 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e426e3-e86f-4efd-8d8c-54e0d780dc7f None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:22.972775 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc47b9-92cf-4b64-8f0f-4067f4132408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:22.973382 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc47b9-92cf-4b64-8f0f-4067f4132408 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:22.973666 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc47b9-92cf-4b64-8f0f-4067f4132408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:22.973938 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc47b9-92cf-4b64-8f0f-4067f4132408 admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:22.975282 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cc47b9-92cf-4b64-8f0f-4067f4132408 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:22.977227 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cc47b9-92cf-4b64-8f0f-4067f4132408 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:23.080525 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1269/2524] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:57:22 2022] POST /identity/v3/projects => generated 375 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:57:23.083640 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159bf7b-edf9-461e-bfdb-8abf79b5aae2 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:23.094194 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159bf7b-edf9-461e-bfdb-8abf79b5aae2 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:23.151020 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665201e-7377-4667-8170-bdf33e939e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:23.151657 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665201e-7377-4667-8170-bdf33e939e5e admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:23.151940 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665201e-7377-4667-8170-bdf33e939e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:23.152219 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665201e-7377-4667-8170-bdf33e939e5e admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:23.152904 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2665201e-7377-4667-8170-bdf33e939e5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:23.154832 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2665201e-7377-4667-8170-bdf33e939e5e admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:23.250247 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1256/2525] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:57:23 2022] POST /identity/v3/users => generated 288 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:57:23.254446 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc47b9-92cf-4b64-8f0f-4067f4132408 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:23.265744 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc47b9-92cf-4b64-8f0f-4067f4132408 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:23.270412 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17074a-a454-4eda-8711-bf9a76af992a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:23.272126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17074a-a454-4eda-8711-bf9a76af992a admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:23.272453 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17074a-a454-4eda-8711-bf9a76af992a admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:23.272738 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17074a-a454-4eda-8711-bf9a76af992a admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:23.273385 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b17074a-a454-4eda-8711-bf9a76af992a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:23.275000 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b17074a-a454-4eda-8711-bf9a76af992a admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:23.285147 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1270/2526] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:57:23.289898 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665201e-7377-4667-8170-bdf33e939e5e admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:23.300371 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665201e-7377-4667-8170-bdf33e939e5e admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:23.305695 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f88f41e-10a5-4cc5-a538-30a952d09413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:23.306300 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88f41e-10a5-4cc5-a538-30a952d09413 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:23.306579 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88f41e-10a5-4cc5-a538-30a952d09413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:23.306856 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88f41e-10a5-4cc5-a538-30a952d09413 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:23.307532 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f88f41e-10a5-4cc5-a538-30a952d09413 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:23.309161 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f88f41e-10a5-4cc5-a538-30a952d09413 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:23.310506 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:23.320873 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1257/2527] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:57:23.328516 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17074a-a454-4eda-8711-bf9a76af992a admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:23.339608 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17074a-a454-4eda-8711-bf9a76af992a admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:23.344368 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46953a1-021b-41c0-886a-64b89edd6f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:23.345059 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46953a1-021b-41c0-886a-64b89edd6f83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:23.345347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46953a1-021b-41c0-886a-64b89edd6f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:23.345628 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46953a1-021b-41c0-886a-64b89edd6f83 admin admin] PATH_INFO: `/v3/projects/b6427769a26448539f830d2fe90d3cb1/users/201f5fd9c18247058d18226a07748242/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:23.360706 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46953a1-021b-41c0-886a-64b89edd6f83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6427769a26448539f830d2fe90d3cb1, user_id=201f5fd9c18247058d18226a07748242, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:23.363610 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46953a1-021b-41c0-886a-64b89edd6f83 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:23.427788 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1271/2528] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:57:23 2022] PUT /identity/v3/projects/b6427769a26448539f830d2fe90d3cb1/users/201f5fd9c18247058d18226a07748242/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:57:23.438570 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18d95c-516d-4a91-ab6b-ac20182ee4d8 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:23.438855 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18d95c-516d-4a91-ab6b-ac20182ee4d8 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:23.439153 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18d95c-516d-4a91-ab6b-ac20182ee4d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:23.467665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-fc18d95c-516d-4a91-ab6b-ac20182ee4d8 None None] MFA Rules not processed for user `201f5fd9c18247058d18226a07748242`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:23.468896 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-fc18d95c-516d-4a91-ab6b-ac20182ee4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:23.531379 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1258/2529] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:57:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:57:23.535888 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46953a1-021b-41c0-886a-64b89edd6f83 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:23.597445 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46953a1-021b-41c0-886a-64b89edd6f83 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:23.602299 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4313b3c2-c1ba-4214-8d9d-a4b448ce3257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:23.602897 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4313b3c2-c1ba-4214-8d9d-a4b448ce3257 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:23.603176 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4313b3c2-c1ba-4214-8d9d-a4b448ce3257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:23.603450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4313b3c2-c1ba-4214-8d9d-a4b448ce3257 admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:23.604080 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4313b3c2-c1ba-4214-8d9d-a4b448ce3257 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:23.606018 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4313b3c2-c1ba-4214-8d9d-a4b448ce3257 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:23.709093 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1272/2530] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:57:23 2022] POST /identity/v3/projects => generated 377 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:57:23.713538 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc18d95c-516d-4a91-ab6b-ac20182ee4d8 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:23.724180 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc18d95c-516d-4a91-ab6b-ac20182ee4d8 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:23.779789 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a8551-b1b1-44c6-8b3a-5b6d2924cb1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:23.780459 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a8551-b1b1-44c6-8b3a-5b6d2924cb1e admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:23.780664 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a8551-b1b1-44c6-8b3a-5b6d2924cb1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:23.780937 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a8551-b1b1-44c6-8b3a-5b6d2924cb1e admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:23.781645 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120a8551-b1b1-44c6-8b3a-5b6d2924cb1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:23.784117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120a8551-b1b1-44c6-8b3a-5b6d2924cb1e admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:23.890356 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1259/2531] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:57:23 2022] POST /identity/v3/users => generated 295 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:57:23.894434 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4313b3c2-c1ba-4214-8d9d-a4b448ce3257 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:23.905173 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4313b3c2-c1ba-4214-8d9d-a4b448ce3257 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:23.910032 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea5fdd-4587-4a01-8767-3751662d368e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:23.910637 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea5fdd-4587-4a01-8767-3751662d368e admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:23.910922 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea5fdd-4587-4a01-8767-3751662d368e admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:23.911199 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea5fdd-4587-4a01-8767-3751662d368e admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:23.913071 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea5fdd-4587-4a01-8767-3751662d368e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:23.914704 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea5fdd-4587-4a01-8767-3751662d368e admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:23.926430 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1273/2532] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:57:23.931636 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a8551-b1b1-44c6-8b3a-5b6d2924cb1e admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:23.941714 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a8551-b1b1-44c6-8b3a-5b6d2924cb1e admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:23.947102 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eebe2b1-8664-4471-8b89-77d44cf826b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:23.947744 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebe2b1-8664-4471-8b89-77d44cf826b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:23.948052 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebe2b1-8664-4471-8b89-77d44cf826b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:23.948326 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebe2b1-8664-4471-8b89-77d44cf826b2 admin admin] PATH_INFO: `/v3/projects/d0f1ba6625144d21965dffe1094b5061/users/d0927f62d7ae47f7898752fa751be338/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:23.962977 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eebe2b1-8664-4471-8b89-77d44cf826b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0f1ba6625144d21965dffe1094b5061, user_id=d0927f62d7ae47f7898752fa751be338, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:23.965716 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eebe2b1-8664-4471-8b89-77d44cf826b2 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:24.040781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1260/2533] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:57:23 2022] PUT /identity/v3/projects/d0f1ba6625144d21965dffe1094b5061/users/d0927f62d7ae47f7898752fa751be338/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:57:24.044670 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea5fdd-4587-4a01-8767-3751662d368e admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:24.056647 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea5fdd-4587-4a01-8767-3751662d368e admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:24.086016 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:24.114658 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fec0d5-787f-406a-870d-0f95243dbd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:24.115967 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fec0d5-787f-406a-870d-0f95243dbd8d admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:24.116245 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fec0d5-787f-406a-870d-0f95243dbd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:24.116559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fec0d5-787f-406a-870d-0f95243dbd8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:24.117211 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fec0d5-787f-406a-870d-0f95243dbd8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:24.119064 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fec0d5-787f-406a-870d-0f95243dbd8d admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:24.129219 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1274/2534] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:57:24 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:57:24.133396 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eebe2b1-8664-4471-8b89-77d44cf826b2 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:24.143851 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eebe2b1-8664-4471-8b89-77d44cf826b2 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:24.148790 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae9f66-bd70-4c39-959e-6f88d909c04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:24.149436 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae9f66-bd70-4c39-959e-6f88d909c04f admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:24.149716 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae9f66-bd70-4c39-959e-6f88d909c04f admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:24.149996 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae9f66-bd70-4c39-959e-6f88d909c04f admin admin] PATH_INFO: `/v3/projects/d0f1ba6625144d21965dffe1094b5061/users/d0927f62d7ae47f7898752fa751be338/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:24.151908 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ae9f66-bd70-4c39-959e-6f88d909c04f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0f1ba6625144d21965dffe1094b5061, user_id=d0927f62d7ae47f7898752fa751be338, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:24.154642 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ae9f66-bd70-4c39-959e-6f88d909c04f admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:24.166006 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1261/2535] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:57:24 2022] PUT /identity/v3/projects/d0f1ba6625144d21965dffe1094b5061/users/d0927f62d7ae47f7898752fa751be338/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:57:24.176619 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d38f0-5c68-4909-9238-04cccaf0b3ad None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:24.176906 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d38f0-5c68-4909-9238-04cccaf0b3ad None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:24.177941 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d38f0-5c68-4909-9238-04cccaf0b3ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:24.207947 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-231d38f0-5c68-4909-9238-04cccaf0b3ad None None] MFA Rules not processed for user `d0927f62d7ae47f7898752fa751be338`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:24.209205 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-231d38f0-5c68-4909-9238-04cccaf0b3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:24.278736 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1275/2536] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:57:24 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:57:24.291964 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae9f66-bd70-4c39-959e-6f88d909c04f admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:24.293613 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-51ae9f66-bd70-4c39-959e-6f88d909c04f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:24.336855 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:24.352152 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae9f66-bd70-4c39-959e-6f88d909c04f admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:24.355831 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e545c-1619-4ac6-be8d-d3b6c854895d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:24.356452 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e545c-1619-4ac6-be8d-d3b6c854895d service cinder] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:24.356741 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e545c-1619-4ac6-be8d-d3b6c854895d service cinder] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:24.357017 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e545c-1619-4ac6-be8d-d3b6c854895d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:24.414574 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1e545c-1619-4ac6-be8d-d3b6c854895d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:24.416015 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1e545c-1619-4ac6-be8d-d3b6c854895d service cinder] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:24.425571 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1262/2537] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:57:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:57:32.811126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d38f0-5c68-4909-9238-04cccaf0b3ad None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:32.812605 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-231d38f0-5c68-4909-9238-04cccaf0b3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:32.825243 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d38f0-5c68-4909-9238-04cccaf0b3ad None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:32.828875 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95820d9e-b2e9-46fa-bf4e-4ae984050c2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:32.829580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95820d9e-b2e9-46fa-bf4e-4ae984050c2e service cinder] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:32.829867 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95820d9e-b2e9-46fa-bf4e-4ae984050c2e service cinder] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:32.830152 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95820d9e-b2e9-46fa-bf4e-4ae984050c2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:32.840390 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95820d9e-b2e9-46fa-bf4e-4ae984050c2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:32.841849 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95820d9e-b2e9-46fa-bf4e-4ae984050c2e service cinder] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:32.852766 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1276/2538] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:57:32 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:57:34.421286 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=31 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:42.847319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=55 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:46.380042 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e545c-1619-4ac6-be8d-d3b6c854895d service cinder] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:46.381737 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-bb1e545c-1619-4ac6-be8d-d3b6c854895d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:46.454027 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e545c-1619-4ac6-be8d-d3b6c854895d service cinder] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:46.460024 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1aa513-99ce-48f3-bc92-fb514c4071f6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6778b7c5ad19484b877494f143050405', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:46.460683 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1aa513-99ce-48f3-bc92-fb514c4071f6 service placement] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:46.460967 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1aa513-99ce-48f3-bc92-fb514c4071f6 service placement] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:46.461273 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1aa513-99ce-48f3-bc92-fb514c4071f6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:46.463065 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-7a1aa513-99ce-48f3-bc92-fb514c4071f6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:46.475201 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1aa513-99ce-48f3-bc92-fb514c4071f6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:46.476824 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1aa513-99ce-48f3-bc92-fb514c4071f6 service placement] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:46.487378 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1263/2539] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:57:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:57:56.482772 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=32 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:09.181523 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95820d9e-b2e9-46fa-bf4e-4ae984050c2e service cinder] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:09.244448 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95820d9e-b2e9-46fa-bf4e-4ae984050c2e service cinder] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:09.249070 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e111d16-ab52-4a9a-a911-ca5e1584fbf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:09.249695 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e111d16-ab52-4a9a-a911-ca5e1584fbf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:09.249974 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e111d16-ab52-4a9a-a911-ca5e1584fbf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:09.250246 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e111d16-ab52-4a9a-a911-ca5e1584fbf7 admin admin] PATH_INFO: `/v3/users/201f5fd9c18247058d18226a07748242` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:09.251369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e111d16-ab52-4a9a-a911-ca5e1584fbf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=201f5fd9c18247058d18226a07748242)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:09.253317 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e111d16-ab52-4a9a-a911-ca5e1584fbf7 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:09.451413 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-4e111d16-ab52-4a9a-a911-ca5e1584fbf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '201f5fd9c18247058d18226a07748242'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:09.506357 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-4e111d16-ab52-4a9a-a911-ca5e1584fbf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '201f5fd9c18247058d18226a07748242'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:09.514027 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-4e111d16-ab52-4a9a-a911-ca5e1584fbf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '201f5fd9c18247058d18226a07748242'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:09.527038 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-4e111d16-ab52-4a9a-a911-ca5e1584fbf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '201f5fd9c18247058d18226a07748242'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:09.527720 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-4e111d16-ab52-4a9a-a911-ca5e1584fbf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '201f5fd9c18247058d18226a07748242'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:09.533765 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1277/2540] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:58:09 2022] DELETE /identity/v3/users/201f5fd9c18247058d18226a07748242 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:58:09.539353 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0357ef-f76b-4787-b788-2e4501dd5905 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:09.539860 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0357ef-f76b-4787-b788-2e4501dd5905 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:09.539976 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0357ef-f76b-4787-b788-2e4501dd5905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:09.558673 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-ed0357ef-f76b-4787-b788-2e4501dd5905 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:09.559873 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-ed0357ef-f76b-4787-b788-2e4501dd5905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:09.620240 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1264/2541] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:58:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:58:09.630502 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e111d16-ab52-4a9a-a911-ca5e1584fbf7 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:09.631953 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-4e111d16-ab52-4a9a-a911-ca5e1584fbf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:09.690743 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e111d16-ab52-4a9a-a911-ca5e1584fbf7 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:09.694962 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e22ab-91de-48ac-be51-7fb2eec1837b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:09.695572 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e22ab-91de-48ac-be51-7fb2eec1837b service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:09.695855 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e22ab-91de-48ac-be51-7fb2eec1837b service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:09.696132 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e22ab-91de-48ac-be51-7fb2eec1837b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:09.705266 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e22ab-91de-48ac-be51-7fb2eec1837b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:09.706674 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e22ab-91de-48ac-be51-7fb2eec1837b service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:09.718205 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1278/2542] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:58:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:58:10.147703 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0357ef-f76b-4787-b788-2e4501dd5905 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:10.149344 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-ed0357ef-f76b-4787-b788-2e4501dd5905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:10.161550 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0357ef-f76b-4787-b788-2e4501dd5905 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:10.166430 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e8efb0-e337-4854-9536-e933dd51e104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:10.166956 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8efb0-e337-4854-9536-e933dd51e104 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:10.167235 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8efb0-e337-4854-9536-e933dd51e104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:10.167508 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8efb0-e337-4854-9536-e933dd51e104 admin admin] PATH_INFO: `/v3/projects/b6427769a26448539f830d2fe90d3cb1` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:10.169438 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e8efb0-e337-4854-9536-e933dd51e104 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6427769a26448539f830d2fe90d3cb1)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:10.171559 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e8efb0-e337-4854-9536-e933dd51e104 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:10.303064 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-74e8efb0-e337-4854-9536-e933dd51e104 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6427769a26448539f830d2fe90d3cb1', 'request_id': 'req-74e8efb0-e337-4854-9536-e933dd51e104'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:10.309389 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-74e8efb0-e337-4854-9536-e933dd51e104 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6427769a26448539f830d2fe90d3cb1', 'request_id': 'req-74e8efb0-e337-4854-9536-e933dd51e104'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:10.386362 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-74e8efb0-e337-4854-9536-e933dd51e104 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6427769a26448539f830d2fe90d3cb1', 'request_id': 'req-74e8efb0-e337-4854-9536-e933dd51e104'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:10.397220 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-74e8efb0-e337-4854-9536-e933dd51e104 admin admin] The token cache is being invalidate because project b6427769a26448539f830d2fe90d3cb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:58:10.397506 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-74e8efb0-e337-4854-9536-e933dd51e104 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:10.398149 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-74e8efb0-e337-4854-9536-e933dd51e104 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:10.399745 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1265/2543] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:58:10 2022] DELETE /identity/v3/projects/b6427769a26448539f830d2fe90d3cb1 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:58:10.403884 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e22ab-91de-48ac-be51-7fb2eec1837b service neutron] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:10.414218 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e22ab-91de-48ac-be51-7fb2eec1837b service neutron] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:10.433268 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:10.471649 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56af3f6f-9e23-41d8-9f92-12e1e1773a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:10.472925 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af3f6f-9e23-41d8-9f92-12e1e1773a0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:10.473315 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af3f6f-9e23-41d8-9f92-12e1e1773a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:10.473569 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af3f6f-9e23-41d8-9f92-12e1e1773a0b admin admin] PATH_INFO: `/v3/users/d0927f62d7ae47f7898752fa751be338` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:10.474718 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56af3f6f-9e23-41d8-9f92-12e1e1773a0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0927f62d7ae47f7898752fa751be338)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:10.476691 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56af3f6f-9e23-41d8-9f92-12e1e1773a0b admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:10.629189 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-56af3f6f-9e23-41d8-9f92-12e1e1773a0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0927f62d7ae47f7898752fa751be338'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:10.695293 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-56af3f6f-9e23-41d8-9f92-12e1e1773a0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0927f62d7ae47f7898752fa751be338'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:10.703879 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-56af3f6f-9e23-41d8-9f92-12e1e1773a0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0927f62d7ae47f7898752fa751be338'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:10.718026 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-56af3f6f-9e23-41d8-9f92-12e1e1773a0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0927f62d7ae47f7898752fa751be338'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:10.720691 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-56af3f6f-9e23-41d8-9f92-12e1e1773a0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0927f62d7ae47f7898752fa751be338'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:10.726596 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1279/2544] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:58:10 2022] DELETE /identity/v3/users/d0927f62d7ae47f7898752fa751be338 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:58:11.136947 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e8efb0-e337-4854-9536-e933dd51e104 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:11.138346 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-74e8efb0-e337-4854-9536-e933dd51e104 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:11.153150 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e8efb0-e337-4854-9536-e933dd51e104 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:11.218297 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e811819-05db-40c3-a7ac-28b05b7c0d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:11.219234 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e811819-05db-40c3-a7ac-28b05b7c0d75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:11.219234 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e811819-05db-40c3-a7ac-28b05b7c0d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:11.219518 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e811819-05db-40c3-a7ac-28b05b7c0d75 admin admin] PATH_INFO: `/v3/projects/d0f1ba6625144d21965dffe1094b5061` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:11.220658 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e811819-05db-40c3-a7ac-28b05b7c0d75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0f1ba6625144d21965dffe1094b5061)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:11.222641 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e811819-05db-40c3-a7ac-28b05b7c0d75 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:11.319835 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:11.336803 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-8e811819-05db-40c3-a7ac-28b05b7c0d75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0f1ba6625144d21965dffe1094b5061', 'request_id': 'req-8e811819-05db-40c3-a7ac-28b05b7c0d75'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:11.342918 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-8e811819-05db-40c3-a7ac-28b05b7c0d75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0f1ba6625144d21965dffe1094b5061', 'request_id': 'req-8e811819-05db-40c3-a7ac-28b05b7c0d75'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:11.414892 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-8e811819-05db-40c3-a7ac-28b05b7c0d75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0f1ba6625144d21965dffe1094b5061', 'request_id': 'req-8e811819-05db-40c3-a7ac-28b05b7c0d75'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:11.426322 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-8e811819-05db-40c3-a7ac-28b05b7c0d75 admin admin] The token cache is being invalidate because project d0f1ba6625144d21965dffe1094b5061 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:58:11.426938 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-8e811819-05db-40c3-a7ac-28b05b7c0d75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:11.428077 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-8e811819-05db-40c3-a7ac-28b05b7c0d75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:11.429702 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1266/2545] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:58:11 2022] DELETE /identity/v3/projects/d0f1ba6625144d21965dffe1094b5061 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:58:11.435733 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c798f727-72e5-4147-9430-149248451060 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:11.436216 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c798f727-72e5-4147-9430-149248451060 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:11.436424 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c798f727-72e5-4147-9430-149248451060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:11.457853 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-c798f727-72e5-4147-9430-149248451060 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:11.459083 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-c798f727-72e5-4147-9430-149248451060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:11.519865 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1280/2546] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:58:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:58:11.528086 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e9770-de95-4af4-b513-65aaac51605d None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:11.528369 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e9770-de95-4af4-b513-65aaac51605d None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:11.528680 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e9770-de95-4af4-b513-65aaac51605d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:11.548392 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-b78e9770-de95-4af4-b513-65aaac51605d None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:11.549632 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-b78e9770-de95-4af4-b513-65aaac51605d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:11.558408 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1267/2547] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:58:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:58:11.563254 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c798f727-72e5-4147-9430-149248451060 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:11.573182 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c798f727-72e5-4147-9430-149248451060 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:11.577752 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab7f28-ddeb-4795-9a82-f3a3d2844206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:11.578345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab7f28-ddeb-4795-9a82-f3a3d2844206 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:11.578621 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab7f28-ddeb-4795-9a82-f3a3d2844206 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:11.578892 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab7f28-ddeb-4795-9a82-f3a3d2844206 admin admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:11.579538 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ab7f28-ddeb-4795-9a82-f3a3d2844206 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:11.579837 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ab7f28-ddeb-4795-9a82-f3a3d2844206 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:11.581535 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ab7f28-ddeb-4795-9a82-f3a3d2844206 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:11.594672 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1281/2548] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:58:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:11.599245 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e9770-de95-4af4-b513-65aaac51605d None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:11.608522 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e9770-de95-4af4-b513-65aaac51605d None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:11.615582 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3d2dc-21a4-4ba8-85fd-37f628ece5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:11.615582 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3d2dc-21a4-4ba8-85fd-37f628ece5d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:11.615582 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3d2dc-21a4-4ba8-85fd-37f628ece5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:11.615582 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3d2dc-21a4-4ba8-85fd-37f628ece5d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:11.616064 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d3d2dc-21a4-4ba8-85fd-37f628ece5d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:11.617534 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d3d2dc-21a4-4ba8-85fd-37f628ece5d6 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:11.692091 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1268/2549] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:58:11 2022] POST /identity/v3/projects => generated 381 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:58:11.696208 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab7f28-ddeb-4795-9a82-f3a3d2844206 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:11.707146 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab7f28-ddeb-4795-9a82-f3a3d2844206 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:11.734541 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:11.765346 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f5dcd-e62d-4efa-8657-43f9fa2e5909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:11.765960 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f5dcd-e62d-4efa-8657-43f9fa2e5909 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:11.766244 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f5dcd-e62d-4efa-8657-43f9fa2e5909 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:11.766530 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f5dcd-e62d-4efa-8657-43f9fa2e5909 admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:11.767167 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f5dcd-e62d-4efa-8657-43f9fa2e5909 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:11.769731 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f5dcd-e62d-4efa-8657-43f9fa2e5909 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:11.889921 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1282/2550] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:58:11 2022] POST /identity/v3/users => generated 291 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:58:11.894489 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3d2dc-21a4-4ba8-85fd-37f628ece5d6 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:11.904953 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3d2dc-21a4-4ba8-85fd-37f628ece5d6 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:11.909486 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02170bcb-ecfc-4646-8da9-46be39d160b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:11.910665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02170bcb-ecfc-4646-8da9-46be39d160b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:11.911125 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02170bcb-ecfc-4646-8da9-46be39d160b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:11.911321 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02170bcb-ecfc-4646-8da9-46be39d160b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:11.911964 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02170bcb-ecfc-4646-8da9-46be39d160b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:11.913586 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02170bcb-ecfc-4646-8da9-46be39d160b3 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:11.923305 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1269/2551] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:58:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:58:11.927615 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f5dcd-e62d-4efa-8657-43f9fa2e5909 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:11.938866 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f5dcd-e62d-4efa-8657-43f9fa2e5909 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:11.944614 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cdad71-28ef-4a74-9909-ee25c560d419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:11.945179 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cdad71-28ef-4a74-9909-ee25c560d419 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:11.945466 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cdad71-28ef-4a74-9909-ee25c560d419 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:11.945745 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cdad71-28ef-4a74-9909-ee25c560d419 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:11.946394 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cdad71-28ef-4a74-9909-ee25c560d419 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:11.948011 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cdad71-28ef-4a74-9909-ee25c560d419 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:11.959285 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1283/2552] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:58:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:58:11.963614 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02170bcb-ecfc-4646-8da9-46be39d160b3 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:11.974031 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02170bcb-ecfc-4646-8da9-46be39d160b3 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:11.978517 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e4504-19f3-4dc5-8537-32f01a4061bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:11.979160 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e4504-19f3-4dc5-8537-32f01a4061bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:11.979441 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e4504-19f3-4dc5-8537-32f01a4061bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:11.979716 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e4504-19f3-4dc5-8537-32f01a4061bf admin admin] PATH_INFO: `/v3/projects/7d783bc389304eff822a14b1376b7454/users/1c2927815ac54b808e33d804deae6f6d/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:11.994302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69e4504-19f3-4dc5-8537-32f01a4061bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d783bc389304eff822a14b1376b7454, user_id=1c2927815ac54b808e33d804deae6f6d, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:11.996965 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69e4504-19f3-4dc5-8537-32f01a4061bf admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:12.065744 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1270/2553] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:58:11 2022] PUT /identity/v3/projects/7d783bc389304eff822a14b1376b7454/users/1c2927815ac54b808e33d804deae6f6d/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:58:12.085104 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffaafb8-e436-4610-9545-c37f5315b5bf None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:12.085386 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffaafb8-e436-4610-9545-c37f5315b5bf None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:12.085686 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffaafb8-e436-4610-9545-c37f5315b5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:12.114053 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-4ffaafb8-e436-4610-9545-c37f5315b5bf None None] MFA Rules not processed for user `1c2927815ac54b808e33d804deae6f6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:12.115277 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-4ffaafb8-e436-4610-9545-c37f5315b5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:12.186479 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1284/2554] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:58:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:58:12.190699 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e4504-19f3-4dc5-8537-32f01a4061bf admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:12.252256 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e4504-19f3-4dc5-8537-32f01a4061bf admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:12.258912 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b5475e-09d8-4451-b4d4-d67f85a09a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:12.260070 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b5475e-09d8-4451-b4d4-d67f85a09a6b admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:12.260624 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b5475e-09d8-4451-b4d4-d67f85a09a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:12.261114 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b5475e-09d8-4451-b4d4-d67f85a09a6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:12.262227 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b5475e-09d8-4451-b4d4-d67f85a09a6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:12.265840 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b5475e-09d8-4451-b4d4-d67f85a09a6b admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:12.358871 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1271/2555] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:58:12 2022] POST /identity/v3/projects => generated 381 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:58:12.363508 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffaafb8-e436-4610-9545-c37f5315b5bf None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:12.374341 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffaafb8-e436-4610-9545-c37f5315b5bf None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:12.434353 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f2309-e539-483d-9987-1888662b62da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:12.434959 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f2309-e539-483d-9987-1888662b62da admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:12.435242 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f2309-e539-483d-9987-1888662b62da admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:12.435521 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f2309-e539-483d-9987-1888662b62da admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:12.436185 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f2309-e539-483d-9987-1888662b62da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:12.438240 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f2309-e539-483d-9987-1888662b62da admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:12.541772 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1285/2556] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:58:12 2022] POST /identity/v3/users => generated 297 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:58:12.545647 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b5475e-09d8-4451-b4d4-d67f85a09a6b admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:12.557183 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b5475e-09d8-4451-b4d4-d67f85a09a6b admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:12.561896 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8235f-a4f8-4f5a-9562-49e80a9592a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:12.562489 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8235f-a4f8-4f5a-9562-49e80a9592a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:12.562770 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8235f-a4f8-4f5a-9562-49e80a9592a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:12.563046 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8235f-a4f8-4f5a-9562-49e80a9592a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:12.563689 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d8235f-a4f8-4f5a-9562-49e80a9592a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:12.565317 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d8235f-a4f8-4f5a-9562-49e80a9592a2 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:12.576211 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1272/2557] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:58:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:58:12.580381 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f2309-e539-483d-9987-1888662b62da admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:12.590651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f2309-e539-483d-9987-1888662b62da admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:12.595268 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275a65e-9a6a-44b1-9aef-06bdc23f389f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:12.595926 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275a65e-9a6a-44b1-9aef-06bdc23f389f admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:12.596204 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275a65e-9a6a-44b1-9aef-06bdc23f389f admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:12.596522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275a65e-9a6a-44b1-9aef-06bdc23f389f admin admin] PATH_INFO: `/v3/projects/47204e9b845747309a436e7655c83902/users/616e2a53f225432c87e21e38390472fa/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:12.611323 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3275a65e-9a6a-44b1-9aef-06bdc23f389f admin admin] RBAC: Authorizing `identity:create_grant(project_id=47204e9b845747309a436e7655c83902, user_id=616e2a53f225432c87e21e38390472fa, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:12.614651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3275a65e-9a6a-44b1-9aef-06bdc23f389f admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:12.718527 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1286/2558] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:58:12 2022] PUT /identity/v3/projects/47204e9b845747309a436e7655c83902/users/616e2a53f225432c87e21e38390472fa/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:58:12.723144 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8235f-a4f8-4f5a-9562-49e80a9592a2 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:12.733190 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8235f-a4f8-4f5a-9562-49e80a9592a2 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:12.754505 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:12.790461 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29fd0d8-fbbf-436f-b5d8-95880e6c3abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:12.791072 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29fd0d8-fbbf-436f-b5d8-95880e6c3abf admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:12.791354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29fd0d8-fbbf-436f-b5d8-95880e6c3abf admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:12.791627 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29fd0d8-fbbf-436f-b5d8-95880e6c3abf admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:12.792323 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29fd0d8-fbbf-436f-b5d8-95880e6c3abf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:12.793897 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29fd0d8-fbbf-436f-b5d8-95880e6c3abf admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:12.804163 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1273/2559] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:58:12 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:58:12.808980 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275a65e-9a6a-44b1-9aef-06bdc23f389f admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:12.814484 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:12.820047 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275a65e-9a6a-44b1-9aef-06bdc23f389f admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:12.824724 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1af7c-33ec-4435-953c-edbfb173fe7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:12.825383 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1af7c-33ec-4435-953c-edbfb173fe7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:12.825671 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1af7c-33ec-4435-953c-edbfb173fe7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:12.825934 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1af7c-33ec-4435-953c-edbfb173fe7a admin admin] PATH_INFO: `/v3/projects/47204e9b845747309a436e7655c83902/users/616e2a53f225432c87e21e38390472fa/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:12.827869 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd1af7c-33ec-4435-953c-edbfb173fe7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=47204e9b845747309a436e7655c83902, user_id=616e2a53f225432c87e21e38390472fa, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:12.830625 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd1af7c-33ec-4435-953c-edbfb173fe7a admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:12.840873 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1287/2560] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:58:12 2022] PUT /identity/v3/projects/47204e9b845747309a436e7655c83902/users/616e2a53f225432c87e21e38390472fa/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:58:12.851712 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46981d2-9032-4944-8a8f-f29d60680f53 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:12.852004 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46981d2-9032-4944-8a8f-f29d60680f53 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:12.852284 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46981d2-9032-4944-8a8f-f29d60680f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:12.881924 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-f46981d2-9032-4944-8a8f-f29d60680f53 None None] MFA Rules not processed for user `616e2a53f225432c87e21e38390472fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:12.883149 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-f46981d2-9032-4944-8a8f-f29d60680f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:12.950534 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1274/2561] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:58:12 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:58:12.963084 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1af7c-33ec-4435-953c-edbfb173fe7a admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:12.964867 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-5dd1af7c-33ec-4435-953c-edbfb173fe7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:13.023908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1af7c-33ec-4435-953c-edbfb173fe7a admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:13.027644 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a114a4cb-dc9b-4565-a1c9-8c23ab6420ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:13.028251 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114a4cb-dc9b-4565-a1c9-8c23ab6420ef service cinder] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:13.028577 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114a4cb-dc9b-4565-a1c9-8c23ab6420ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:13.028857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114a4cb-dc9b-4565-a1c9-8c23ab6420ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:13.038848 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a114a4cb-dc9b-4565-a1c9-8c23ab6420ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:13.040291 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a114a4cb-dc9b-4565-a1c9-8c23ab6420ef service cinder] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:13.049935 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1288/2562] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:58:12 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:58:13.201539 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46981d2-9032-4944-8a8f-f29d60680f53 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:13.213395 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46981d2-9032-4944-8a8f-f29d60680f53 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:13.219552 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cec4fb-0511-4ab8-84d2-bcfffd351b68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:13.220602 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cec4fb-0511-4ab8-84d2-bcfffd351b68 service cinder] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:13.221092 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cec4fb-0511-4ab8-84d2-bcfffd351b68 service cinder] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:13.221384 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cec4fb-0511-4ab8-84d2-bcfffd351b68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:13.279313 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cec4fb-0511-4ab8-84d2-bcfffd351b68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:13.280784 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cec4fb-0511-4ab8-84d2-bcfffd351b68 service cinder] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:13.290843 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1275/2563] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:58:13 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:58:13.794996 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a114a4cb-dc9b-4565-a1c9-8c23ab6420ef service cinder] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:13.878412 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a114a4cb-dc9b-4565-a1c9-8c23ab6420ef service cinder] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:13.884698 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86d523-18e4-43e4-ad5a-a8435a22ec94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:13.885352 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86d523-18e4-43e4-ad5a-a8435a22ec94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:13.885692 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86d523-18e4-43e4-ad5a-a8435a22ec94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:13.887143 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86d523-18e4-43e4-ad5a-a8435a22ec94 admin admin] PATH_INFO: `/v3/users/1c2927815ac54b808e33d804deae6f6d` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:13.892077 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86d523-18e4-43e4-ad5a-a8435a22ec94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c2927815ac54b808e33d804deae6f6d)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:13.895853 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86d523-18e4-43e4-ad5a-a8435a22ec94 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:14.082342 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-6c86d523-18e4-43e4-ad5a-a8435a22ec94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c2927815ac54b808e33d804deae6f6d'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:14.141082 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-6c86d523-18e4-43e4-ad5a-a8435a22ec94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c2927815ac54b808e33d804deae6f6d'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:14.149527 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-6c86d523-18e4-43e4-ad5a-a8435a22ec94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c2927815ac54b808e33d804deae6f6d'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:14.162443 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-6c86d523-18e4-43e4-ad5a-a8435a22ec94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c2927815ac54b808e33d804deae6f6d'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:14.163162 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-6c86d523-18e4-43e4-ad5a-a8435a22ec94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c2927815ac54b808e33d804deae6f6d'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:14.169243 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1289/2564] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:58:13 2022] DELETE /identity/v3/users/1c2927815ac54b808e33d804deae6f6d => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:58:14.174022 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6963f12-e32d-4f31-ac37-3427435e2da5 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:14.174305 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6963f12-e32d-4f31-ac37-3427435e2da5 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:14.174579 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6963f12-e32d-4f31-ac37-3427435e2da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:14.195137 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-f6963f12-e32d-4f31-ac37-3427435e2da5 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:14.196353 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-f6963f12-e32d-4f31-ac37-3427435e2da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:14.236742 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=100 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:14.255744 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1276/2565] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:58:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:58:14.265723 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86d523-18e4-43e4-ad5a-a8435a22ec94 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:14.267163 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-6c86d523-18e4-43e4-ad5a-a8435a22ec94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:14.280935 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:14.327327 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86d523-18e4-43e4-ad5a-a8435a22ec94 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:14.331119 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1951dd-3338-4742-892d-b30a8c5d5ad4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:14.331731 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1951dd-3338-4742-892d-b30a8c5d5ad4 service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:14.332016 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1951dd-3338-4742-892d-b30a8c5d5ad4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:14.332294 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1951dd-3338-4742-892d-b30a8c5d5ad4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:14.343794 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1951dd-3338-4742-892d-b30a8c5d5ad4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:14.348735 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1951dd-3338-4742-892d-b30a8c5d5ad4 service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:14.355893 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1290/2566] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:58:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:58:14.793990 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6963f12-e32d-4f31-ac37-3427435e2da5 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:14.795521 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-f6963f12-e32d-4f31-ac37-3427435e2da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:14.808438 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6963f12-e32d-4f31-ac37-3427435e2da5 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:14.813435 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac8a568-2c54-44af-9d45-ba06d1e50ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:14.814070 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac8a568-2c54-44af-9d45-ba06d1e50ca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:14.814357 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac8a568-2c54-44af-9d45-ba06d1e50ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:14.814634 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac8a568-2c54-44af-9d45-ba06d1e50ca5 admin admin] PATH_INFO: `/v3/projects/7d783bc389304eff822a14b1376b7454` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:14.815819 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac8a568-2c54-44af-9d45-ba06d1e50ca5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d783bc389304eff822a14b1376b7454)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:14.818040 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac8a568-2c54-44af-9d45-ba06d1e50ca5 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:14.936141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-5ac8a568-2c54-44af-9d45-ba06d1e50ca5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d783bc389304eff822a14b1376b7454', 'request_id': 'req-5ac8a568-2c54-44af-9d45-ba06d1e50ca5'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:14.943543 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-5ac8a568-2c54-44af-9d45-ba06d1e50ca5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d783bc389304eff822a14b1376b7454', 'request_id': 'req-5ac8a568-2c54-44af-9d45-ba06d1e50ca5'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:15.003917 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-5ac8a568-2c54-44af-9d45-ba06d1e50ca5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d783bc389304eff822a14b1376b7454', 'request_id': 'req-5ac8a568-2c54-44af-9d45-ba06d1e50ca5'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:15.015020 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-5ac8a568-2c54-44af-9d45-ba06d1e50ca5 admin admin] The token cache is being invalidate because project 7d783bc389304eff822a14b1376b7454 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:58:15.015310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-5ac8a568-2c54-44af-9d45-ba06d1e50ca5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:15.015945 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-5ac8a568-2c54-44af-9d45-ba06d1e50ca5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:15.017472 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1277/2567] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:58:14 2022] DELETE /identity/v3/projects/7d783bc389304eff822a14b1376b7454 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:58:15.022927 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1951dd-3338-4742-892d-b30a8c5d5ad4 service neutron] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:15.032787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1951dd-3338-4742-892d-b30a8c5d5ad4 service neutron] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:15.089350 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c0bb28-0f89-48b0-ae39-4d9f7871deb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:15.089892 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0bb28-0f89-48b0-ae39-4d9f7871deb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:15.090170 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0bb28-0f89-48b0-ae39-4d9f7871deb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:15.090443 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0bb28-0f89-48b0-ae39-4d9f7871deb3 admin admin] PATH_INFO: `/v3/users/616e2a53f225432c87e21e38390472fa` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:15.091565 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c0bb28-0f89-48b0-ae39-4d9f7871deb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=616e2a53f225432c87e21e38390472fa)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:15.093527 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c0bb28-0f89-48b0-ae39-4d9f7871deb3 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:15.305526 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-14c0bb28-0f89-48b0-ae39-4d9f7871deb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '616e2a53f225432c87e21e38390472fa'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:15.375180 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-14c0bb28-0f89-48b0-ae39-4d9f7871deb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '616e2a53f225432c87e21e38390472fa'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:15.382930 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-14c0bb28-0f89-48b0-ae39-4d9f7871deb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '616e2a53f225432c87e21e38390472fa'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:15.395303 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-14c0bb28-0f89-48b0-ae39-4d9f7871deb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '616e2a53f225432c87e21e38390472fa'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:15.395953 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-14c0bb28-0f89-48b0-ae39-4d9f7871deb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '616e2a53f225432c87e21e38390472fa'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:15.401791 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1291/2568] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:58:15 2022] DELETE /identity/v3/users/616e2a53f225432c87e21e38390472fa => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:58:15.805499 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac8a568-2c54-44af-9d45-ba06d1e50ca5 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:15.806917 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-5ac8a568-2c54-44af-9d45-ba06d1e50ca5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:15.822151 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac8a568-2c54-44af-9d45-ba06d1e50ca5 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:15.881862 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1df47e-15da-4a11-bb7b-c3990ddc0d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:15.882492 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1df47e-15da-4a11-bb7b-c3990ddc0d2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:15.882773 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1df47e-15da-4a11-bb7b-c3990ddc0d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:15.883047 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1df47e-15da-4a11-bb7b-c3990ddc0d2c admin admin] PATH_INFO: `/v3/projects/47204e9b845747309a436e7655c83902` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:15.884252 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1df47e-15da-4a11-bb7b-c3990ddc0d2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=47204e9b845747309a436e7655c83902)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:15.886189 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1df47e-15da-4a11-bb7b-c3990ddc0d2c admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:15.992918 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-6b1df47e-15da-4a11-bb7b-c3990ddc0d2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47204e9b845747309a436e7655c83902', 'request_id': 'req-6b1df47e-15da-4a11-bb7b-c3990ddc0d2c'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:15.999176 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-6b1df47e-15da-4a11-bb7b-c3990ddc0d2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47204e9b845747309a436e7655c83902', 'request_id': 'req-6b1df47e-15da-4a11-bb7b-c3990ddc0d2c'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:16.060507 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-6b1df47e-15da-4a11-bb7b-c3990ddc0d2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47204e9b845747309a436e7655c83902', 'request_id': 'req-6b1df47e-15da-4a11-bb7b-c3990ddc0d2c'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:16.067879 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:16.072585 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-6b1df47e-15da-4a11-bb7b-c3990ddc0d2c admin admin] The token cache is being invalidate because project 47204e9b845747309a436e7655c83902 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:58:16.072879 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-6b1df47e-15da-4a11-bb7b-c3990ddc0d2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:16.073526 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-6b1df47e-15da-4a11-bb7b-c3990ddc0d2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:16.075015 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1278/2569] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:58:15 2022] DELETE /identity/v3/projects/47204e9b845747309a436e7655c83902 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:58:16.080102 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65873c-a97d-4ad8-a5c3-c80bcd19dc2e None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:16.080599 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65873c-a97d-4ad8-a5c3-c80bcd19dc2e None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:16.080754 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65873c-a97d-4ad8-a5c3-c80bcd19dc2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:16.102893 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-fb65873c-a97d-4ad8-a5c3-c80bcd19dc2e None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:16.104116 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-fb65873c-a97d-4ad8-a5c3-c80bcd19dc2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:16.134043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:16.169760 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1292/2570] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:58:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:58:16.178719 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df185f-1dde-480c-9659-75e06d38b190 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:16.179152 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df185f-1dde-480c-9659-75e06d38b190 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:16.179568 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df185f-1dde-480c-9659-75e06d38b190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:16.200709 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-c3df185f-1dde-480c-9659-75e06d38b190 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:16.201912 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-c3df185f-1dde-480c-9659-75e06d38b190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:16.210633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1279/2571] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:58:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:58:16.215147 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb65873c-a97d-4ad8-a5c3-c80bcd19dc2e None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:16.225486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb65873c-a97d-4ad8-a5c3-c80bcd19dc2e None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:16.230867 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0c517-3c09-4a38-9c31-3846d1acc042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:16.231527 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0c517-3c09-4a38-9c31-3846d1acc042 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:16.234862 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0c517-3c09-4a38-9c31-3846d1acc042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:16.234862 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0c517-3c09-4a38-9c31-3846d1acc042 admin admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:16.234862 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b0c517-3c09-4a38-9c31-3846d1acc042 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:16.234862 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b0c517-3c09-4a38-9c31-3846d1acc042 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:16.234862 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b0c517-3c09-4a38-9c31-3846d1acc042 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:16.246352 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1293/2572] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:58:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:16.250782 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3df185f-1dde-480c-9659-75e06d38b190 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:16.260652 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3df185f-1dde-480c-9659-75e06d38b190 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:16.265420 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcae087-7165-4603-a477-f852150ab6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:16.266025 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcae087-7165-4603-a477-f852150ab6e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:16.266304 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcae087-7165-4603-a477-f852150ab6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:16.266581 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcae087-7165-4603-a477-f852150ab6e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:16.267216 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcae087-7165-4603-a477-f852150ab6e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:16.269705 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcae087-7165-4603-a477-f852150ab6e2 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:16.376548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1280/2573] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:58:16 2022] POST /identity/v3/projects => generated 377 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:58:16.380921 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0c517-3c09-4a38-9c31-3846d1acc042 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:16.390975 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0c517-3c09-4a38-9c31-3846d1acc042 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:16.450604 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea17fdc1-ca42-4557-8bb1-00915ac7638b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:16.451134 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17fdc1-ca42-4557-8bb1-00915ac7638b admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:16.451450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17fdc1-ca42-4557-8bb1-00915ac7638b admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:16.451726 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17fdc1-ca42-4557-8bb1-00915ac7638b admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:16.452386 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea17fdc1-ca42-4557-8bb1-00915ac7638b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:16.454387 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea17fdc1-ca42-4557-8bb1-00915ac7638b admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:16.528268 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1294/2574] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:58:16 2022] POST /identity/v3/users => generated 289 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:58:16.532445 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcae087-7165-4603-a477-f852150ab6e2 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:16.543254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcae087-7165-4603-a477-f852150ab6e2 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:16.548249 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c2889-2e6d-4bd0-8281-426bf042f0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:16.548894 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c2889-2e6d-4bd0-8281-426bf042f0ac admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:16.549182 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c2889-2e6d-4bd0-8281-426bf042f0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:16.549514 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c2889-2e6d-4bd0-8281-426bf042f0ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:16.550496 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2c2889-2e6d-4bd0-8281-426bf042f0ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:16.552070 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2c2889-2e6d-4bd0-8281-426bf042f0ac admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:16.562612 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1281/2575] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:58:16.567185 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea17fdc1-ca42-4557-8bb1-00915ac7638b admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:16.577136 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea17fdc1-ca42-4557-8bb1-00915ac7638b admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:16.582798 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f6bf0d-2268-4180-bbb4-0389c86aeb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:16.583399 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f6bf0d-2268-4180-bbb4-0389c86aeb3a admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:16.583680 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f6bf0d-2268-4180-bbb4-0389c86aeb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:16.583983 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f6bf0d-2268-4180-bbb4-0389c86aeb3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:16.584663 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f6bf0d-2268-4180-bbb4-0389c86aeb3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:16.586273 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f6bf0d-2268-4180-bbb4-0389c86aeb3a admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:16.596869 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1295/2576] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:58:16.602031 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c2889-2e6d-4bd0-8281-426bf042f0ac admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:16.612241 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c2889-2e6d-4bd0-8281-426bf042f0ac admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:16.618040 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9900355-eaa4-460d-98cc-7dbe07400663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:16.624107 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9900355-eaa4-460d-98cc-7dbe07400663 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:16.624107 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9900355-eaa4-460d-98cc-7dbe07400663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:16.624107 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9900355-eaa4-460d-98cc-7dbe07400663 admin admin] PATH_INFO: `/v3/projects/6a2cb3f81fd34af999d8c1dba151017b/users/ce3fd96dc5164fdfb7f02e8c2e4b639a/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:16.638369 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9900355-eaa4-460d-98cc-7dbe07400663 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a2cb3f81fd34af999d8c1dba151017b, user_id=ce3fd96dc5164fdfb7f02e8c2e4b639a, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:16.641044 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9900355-eaa4-460d-98cc-7dbe07400663 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:16.730072 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1282/2577] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:58:16 2022] PUT /identity/v3/projects/6a2cb3f81fd34af999d8c1dba151017b/users/ce3fd96dc5164fdfb7f02e8c2e4b639a/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:58:16.742180 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94dc492-c682-4e45-95c8-e97c23a064a4 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:16.742467 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94dc492-c682-4e45-95c8-e97c23a064a4 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:16.742745 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94dc492-c682-4e45-95c8-e97c23a064a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:16.770066 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-e94dc492-c682-4e45-95c8-e97c23a064a4 None None] MFA Rules not processed for user `ce3fd96dc5164fdfb7f02e8c2e4b639a`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:16.771267 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-e94dc492-c682-4e45-95c8-e97c23a064a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:16.835302 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1296/2578] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:58:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:58:16.846477 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9900355-eaa4-460d-98cc-7dbe07400663 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:16.848422 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-a9900355-eaa4-460d-98cc-7dbe07400663 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:16.904812 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9900355-eaa4-460d-98cc-7dbe07400663 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:16.908575 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5aaccf-6f94-4b30-9b05-ed82fbee72f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:16.909181 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5aaccf-6f94-4b30-9b05-ed82fbee72f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:16.909615 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5aaccf-6f94-4b30-9b05-ed82fbee72f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:16.910307 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5aaccf-6f94-4b30-9b05-ed82fbee72f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:16.918674 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5aaccf-6f94-4b30-9b05-ed82fbee72f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:16.920096 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5aaccf-6f94-4b30-9b05-ed82fbee72f4 service cinder] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:16.930446 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1283/2579] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:58:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:58:26.832084 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=84,keystone:INSERT=3 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:26.925386 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=54,keystone:INSERT=2 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:49.587044 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94dc492-c682-4e45-95c8-e97c23a064a4 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:49.653520 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94dc492-c682-4e45-95c8-e97c23a064a4 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:49.659927 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33096fe-a54b-4cb9-af02-acc82458abc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:49.660655 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33096fe-a54b-4cb9-af02-acc82458abc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:49.660935 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33096fe-a54b-4cb9-af02-acc82458abc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:49.661212 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33096fe-a54b-4cb9-af02-acc82458abc6 admin admin] PATH_INFO: `/v3/users/ce3fd96dc5164fdfb7f02e8c2e4b639a` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:49.662377 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33096fe-a54b-4cb9-af02-acc82458abc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce3fd96dc5164fdfb7f02e8c2e4b639a)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:49.664312 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33096fe-a54b-4cb9-af02-acc82458abc6 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:49.851355 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-c33096fe-a54b-4cb9-af02-acc82458abc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce3fd96dc5164fdfb7f02e8c2e4b639a'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:49.940429 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-c33096fe-a54b-4cb9-af02-acc82458abc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce3fd96dc5164fdfb7f02e8c2e4b639a'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:49.948814 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-c33096fe-a54b-4cb9-af02-acc82458abc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce3fd96dc5164fdfb7f02e8c2e4b639a'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:49.964549 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-c33096fe-a54b-4cb9-af02-acc82458abc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce3fd96dc5164fdfb7f02e8c2e4b639a'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:49.965305 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-c33096fe-a54b-4cb9-af02-acc82458abc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce3fd96dc5164fdfb7f02e8c2e4b639a'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:49.971879 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1297/2580] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:58:49 2022] DELETE /identity/v3/users/ce3fd96dc5164fdfb7f02e8c2e4b639a => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:58:49.977090 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef8fc1-b05a-4624-97b1-45cffc766121 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:49.977625 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef8fc1-b05a-4624-97b1-45cffc766121 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:49.977625 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef8fc1-b05a-4624-97b1-45cffc766121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:49.997948 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-a7ef8fc1-b05a-4624-97b1-45cffc766121 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:49.999171 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-a7ef8fc1-b05a-4624-97b1-45cffc766121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:50.085912 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1284/2581] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:58:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:58:50.090856 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33096fe-a54b-4cb9-af02-acc82458abc6 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:50.092271 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-c33096fe-a54b-4cb9-af02-acc82458abc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:50.160568 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33096fe-a54b-4cb9-af02-acc82458abc6 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:50.164287 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae8eff-b563-4564-8e7e-9542d78e5afb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:50.164943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae8eff-b563-4564-8e7e-9542d78e5afb service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:50.165304 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae8eff-b563-4564-8e7e-9542d78e5afb service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:50.165583 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae8eff-b563-4564-8e7e-9542d78e5afb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:50.176311 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ae8eff-b563-4564-8e7e-9542d78e5afb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:50.177810 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ae8eff-b563-4564-8e7e-9542d78e5afb service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:50.192387 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1298/2582] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:58:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:58:50.645190 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ef8fc1-b05a-4624-97b1-45cffc766121 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:50.646716 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-a7ef8fc1-b05a-4624-97b1-45cffc766121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:50.659065 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ef8fc1-b05a-4624-97b1-45cffc766121 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:50.670364 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a346cf4c-b087-4e2c-b585-ec3e144c5bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:50.671117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346cf4c-b087-4e2c-b585-ec3e144c5bfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:50.671383 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346cf4c-b087-4e2c-b585-ec3e144c5bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:50.671649 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346cf4c-b087-4e2c-b585-ec3e144c5bfc admin admin] PATH_INFO: `/v3/projects/6a2cb3f81fd34af999d8c1dba151017b` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:50.673272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a346cf4c-b087-4e2c-b585-ec3e144c5bfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a2cb3f81fd34af999d8c1dba151017b)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:50.676054 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a346cf4c-b087-4e2c-b585-ec3e144c5bfc admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:50.834741 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-a346cf4c-b087-4e2c-b585-ec3e144c5bfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a2cb3f81fd34af999d8c1dba151017b', 'request_id': 'req-a346cf4c-b087-4e2c-b585-ec3e144c5bfc'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:50.841117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-a346cf4c-b087-4e2c-b585-ec3e144c5bfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a2cb3f81fd34af999d8c1dba151017b', 'request_id': 'req-a346cf4c-b087-4e2c-b585-ec3e144c5bfc'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:50.917203 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-a346cf4c-b087-4e2c-b585-ec3e144c5bfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a2cb3f81fd34af999d8c1dba151017b', 'request_id': 'req-a346cf4c-b087-4e2c-b585-ec3e144c5bfc'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:50.932779 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-a346cf4c-b087-4e2c-b585-ec3e144c5bfc admin admin] The token cache is being invalidate because project 6a2cb3f81fd34af999d8c1dba151017b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:58:50.933082 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-a346cf4c-b087-4e2c-b585-ec3e144c5bfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:50.933815 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-a346cf4c-b087-4e2c-b585-ec3e144c5bfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:50.935432 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1285/2583] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:58:50 2022] DELETE /identity/v3/projects/6a2cb3f81fd34af999d8c1dba151017b => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:58:50.941286 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba4223-af61-46cd-854e-8f6e62fe4aaa None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:50.942079 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba4223-af61-46cd-854e-8f6e62fe4aaa None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:50.942079 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba4223-af61-46cd-854e-8f6e62fe4aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:50.963602 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-afba4223-af61-46cd-854e-8f6e62fe4aaa None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:50.965009 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-afba4223-af61-46cd-854e-8f6e62fe4aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:50.989879 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:51.042241 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1299/2584] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:58:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:58:51.053235 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4698e-5850-489a-adca-36960ba0ebcf None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:51.053792 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4698e-5850-489a-adca-36960ba0ebcf None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:51.053792 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4698e-5850-489a-adca-36960ba0ebcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:51.074714 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-1bf4698e-5850-489a-adca-36960ba0ebcf None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:51.076039 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-1bf4698e-5850-489a-adca-36960ba0ebcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:51.084767 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1286/2585] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:58:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:58:51.089159 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afba4223-af61-46cd-854e-8f6e62fe4aaa None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:51.101036 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afba4223-af61-46cd-854e-8f6e62fe4aaa None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:51.107814 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5afc0d-a341-469c-b80f-859e1b377aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:51.112016 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5afc0d-a341-469c-b80f-859e1b377aea admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:51.112016 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5afc0d-a341-469c-b80f-859e1b377aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:51.112016 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5afc0d-a341-469c-b80f-859e1b377aea admin admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:51.112016 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5afc0d-a341-469c-b80f-859e1b377aea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:51.112016 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5afc0d-a341-469c-b80f-859e1b377aea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:51.113081 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5afc0d-a341-469c-b80f-859e1b377aea admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:51.128706 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1300/2586] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:58:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:51.131519 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf4698e-5850-489a-adca-36960ba0ebcf None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:51.144379 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf4698e-5850-489a-adca-36960ba0ebcf None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:51.149339 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4715f57-b99e-44d1-b657-8eda629d35ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:51.149960 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4715f57-b99e-44d1-b657-8eda629d35ca admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:51.150240 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4715f57-b99e-44d1-b657-8eda629d35ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:51.150515 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4715f57-b99e-44d1-b657-8eda629d35ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:51.151143 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4715f57-b99e-44d1-b657-8eda629d35ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:51.153096 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4715f57-b99e-44d1-b657-8eda629d35ca admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:51.274851 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1287/2587] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:58:51 2022] POST /identity/v3/projects => generated 371 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:58:51.279083 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5afc0d-a341-469c-b80f-859e1b377aea admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:51.291453 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5afc0d-a341-469c-b80f-859e1b377aea admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:51.354554 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0790bdc6-1302-41d7-bf0f-168c83fcdaf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:51.355171 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790bdc6-1302-41d7-bf0f-168c83fcdaf9 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:51.355468 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790bdc6-1302-41d7-bf0f-168c83fcdaf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:51.355760 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790bdc6-1302-41d7-bf0f-168c83fcdaf9 admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:51.356459 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0790bdc6-1302-41d7-bf0f-168c83fcdaf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:51.358615 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0790bdc6-1302-41d7-bf0f-168c83fcdaf9 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:51.472777 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1301/2588] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:58:51 2022] POST /identity/v3/users => generated 286 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:58:51.476903 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4715f57-b99e-44d1-b657-8eda629d35ca admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:51.487348 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4715f57-b99e-44d1-b657-8eda629d35ca admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:51.492374 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b656a71-8817-4e3f-80d8-be35efac2659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:51.493191 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b656a71-8817-4e3f-80d8-be35efac2659 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:51.493320 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b656a71-8817-4e3f-80d8-be35efac2659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:51.493558 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b656a71-8817-4e3f-80d8-be35efac2659 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:51.494208 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b656a71-8817-4e3f-80d8-be35efac2659 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:51.495841 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b656a71-8817-4e3f-80d8-be35efac2659 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:51.506215 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1288/2589] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:58:51.511335 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0790bdc6-1302-41d7-bf0f-168c83fcdaf9 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:51.522621 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0790bdc6-1302-41d7-bf0f-168c83fcdaf9 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:51.527447 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afebf6c-f842-40db-95b6-845f29a723c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:51.528123 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afebf6c-f842-40db-95b6-845f29a723c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:51.528341 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afebf6c-f842-40db-95b6-845f29a723c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:51.529061 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afebf6c-f842-40db-95b6-845f29a723c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:51.529295 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afebf6c-f842-40db-95b6-845f29a723c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:51.531067 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afebf6c-f842-40db-95b6-845f29a723c6 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:51.543618 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1302/2590] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:58:51.548082 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b656a71-8817-4e3f-80d8-be35efac2659 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:51.558855 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b656a71-8817-4e3f-80d8-be35efac2659 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:51.563405 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9bbc2-39df-4fc3-b76a-b91b1a821aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:51.564181 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9bbc2-39df-4fc3-b76a-b91b1a821aff admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:51.564379 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9bbc2-39df-4fc3-b76a-b91b1a821aff admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:51.564854 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9bbc2-39df-4fc3-b76a-b91b1a821aff admin admin] PATH_INFO: `/v3/projects/f945121d9e704a558568dc7b000831c6/users/e8fd5abd94314e8c9a1675641454f118/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:51.579226 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe9bbc2-39df-4fc3-b76a-b91b1a821aff admin admin] RBAC: Authorizing `identity:create_grant(project_id=f945121d9e704a558568dc7b000831c6, user_id=e8fd5abd94314e8c9a1675641454f118, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:51.582634 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe9bbc2-39df-4fc3-b76a-b91b1a821aff admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:51.666607 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1289/2591] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:58:51 2022] PUT /identity/v3/projects/f945121d9e704a558568dc7b000831c6/users/e8fd5abd94314e8c9a1675641454f118/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:58:51.684627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2453b9-08a8-4413-80e3-1ba7217ac731 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:51.684908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2453b9-08a8-4413-80e3-1ba7217ac731 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:51.685184 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2453b9-08a8-4413-80e3-1ba7217ac731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:51.715596 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-7e2453b9-08a8-4413-80e3-1ba7217ac731 None None] MFA Rules not processed for user `e8fd5abd94314e8c9a1675641454f118`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:51.716853 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-7e2453b9-08a8-4413-80e3-1ba7217ac731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:51.781821 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1303/2592] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:58:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:58:51.793638 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9bbc2-39df-4fc3-b76a-b91b1a821aff admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:51.795397 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-0fe9bbc2-39df-4fc3-b76a-b91b1a821aff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:51.817781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:51.854528 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9bbc2-39df-4fc3-b76a-b91b1a821aff admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:51.858309 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18afce-d8d5-4964-bfed-c5bcad06cc66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:51.858908 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18afce-d8d5-4964-bfed-c5bcad06cc66 service cinder] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:51.873781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18afce-d8d5-4964-bfed-c5bcad06cc66 service cinder] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:51.873781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18afce-d8d5-4964-bfed-c5bcad06cc66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:51.873781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e18afce-d8d5-4964-bfed-c5bcad06cc66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:51.873781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e18afce-d8d5-4964-bfed-c5bcad06cc66 service cinder] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:51.878906 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1290/2593] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:58:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:58:51.905315 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2453b9-08a8-4413-80e3-1ba7217ac731 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:51.926224 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:51.978313 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2453b9-08a8-4413-80e3-1ba7217ac731 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:51.983144 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966f70bd-454a-4df3-a27f-5ff2fa02421b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:51.983778 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966f70bd-454a-4df3-a27f-5ff2fa02421b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:51.984063 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966f70bd-454a-4df3-a27f-5ff2fa02421b admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:51.984334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966f70bd-454a-4df3-a27f-5ff2fa02421b admin admin] PATH_INFO: `/v3/users/e8fd5abd94314e8c9a1675641454f118` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:51.985577 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966f70bd-454a-4df3-a27f-5ff2fa02421b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8fd5abd94314e8c9a1675641454f118)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:51.988075 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966f70bd-454a-4df3-a27f-5ff2fa02421b admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:52.159717 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-966f70bd-454a-4df3-a27f-5ff2fa02421b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8fd5abd94314e8c9a1675641454f118'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:52.220332 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-966f70bd-454a-4df3-a27f-5ff2fa02421b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8fd5abd94314e8c9a1675641454f118'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:52.227949 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-966f70bd-454a-4df3-a27f-5ff2fa02421b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8fd5abd94314e8c9a1675641454f118'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:52.240243 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-966f70bd-454a-4df3-a27f-5ff2fa02421b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8fd5abd94314e8c9a1675641454f118'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:52.240963 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-966f70bd-454a-4df3-a27f-5ff2fa02421b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8fd5abd94314e8c9a1675641454f118'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:52.247193 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1304/2594] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:58:51 2022] DELETE /identity/v3/users/e8fd5abd94314e8c9a1675641454f118 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:58:52.252709 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1909eb5c-b5b9-480a-bbd3-8c13f42c14f0 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:52.253210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1909eb5c-b5b9-480a-bbd3-8c13f42c14f0 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:52.253378 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1909eb5c-b5b9-480a-bbd3-8c13f42c14f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:52.272099 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-1909eb5c-b5b9-480a-bbd3-8c13f42c14f0 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:52.273346 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-1909eb5c-b5b9-480a-bbd3-8c13f42c14f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:52.334080 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1291/2595] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:58:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:58:52.343704 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966f70bd-454a-4df3-a27f-5ff2fa02421b admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:52.345391 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-966f70bd-454a-4df3-a27f-5ff2fa02421b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:52.403595 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966f70bd-454a-4df3-a27f-5ff2fa02421b admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:52.407308 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24060c53-3f2b-44e5-859e-dda82583901d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:52.410102 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24060c53-3f2b-44e5-859e-dda82583901d service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:52.410102 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24060c53-3f2b-44e5-859e-dda82583901d service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:52.410102 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24060c53-3f2b-44e5-859e-dda82583901d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:52.418433 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24060c53-3f2b-44e5-859e-dda82583901d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:52.419893 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24060c53-3f2b-44e5-859e-dda82583901d service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:52.430585 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1305/2596] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:58:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:58:52.816219 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1909eb5c-b5b9-480a-bbd3-8c13f42c14f0 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:52.817719 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-1909eb5c-b5b9-480a-bbd3-8c13f42c14f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:52.834614 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1909eb5c-b5b9-480a-bbd3-8c13f42c14f0 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:52.840305 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f96ba2f-9da5-4843-b2f5-30405c494092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:52.840984 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96ba2f-9da5-4843-b2f5-30405c494092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:52.841293 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96ba2f-9da5-4843-b2f5-30405c494092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:52.841560 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96ba2f-9da5-4843-b2f5-30405c494092 admin admin] PATH_INFO: `/v3/projects/f945121d9e704a558568dc7b000831c6` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:52.842695 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f96ba2f-9da5-4843-b2f5-30405c494092 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f945121d9e704a558568dc7b000831c6)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:52.844634 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f96ba2f-9da5-4843-b2f5-30405c494092 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:52.997414 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-4f96ba2f-9da5-4843-b2f5-30405c494092 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f945121d9e704a558568dc7b000831c6', 'request_id': 'req-4f96ba2f-9da5-4843-b2f5-30405c494092'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:53.003788 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-4f96ba2f-9da5-4843-b2f5-30405c494092 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f945121d9e704a558568dc7b000831c6', 'request_id': 'req-4f96ba2f-9da5-4843-b2f5-30405c494092'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:53.077580 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-4f96ba2f-9da5-4843-b2f5-30405c494092 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f945121d9e704a558568dc7b000831c6', 'request_id': 'req-4f96ba2f-9da5-4843-b2f5-30405c494092'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:53.098030 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-4f96ba2f-9da5-4843-b2f5-30405c494092 admin admin] The token cache is being invalidate because project f945121d9e704a558568dc7b000831c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:58:53.098377 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-4f96ba2f-9da5-4843-b2f5-30405c494092 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:53.099019 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-4f96ba2f-9da5-4843-b2f5-30405c494092 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:53.100582 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1292/2597] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:58:52 2022] DELETE /identity/v3/projects/f945121d9e704a558568dc7b000831c6 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:58:53.105722 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da5f9b-49c6-46e2-82de-2c89ee74bed6 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:53.106208 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da5f9b-49c6-46e2-82de-2c89ee74bed6 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:53.106363 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da5f9b-49c6-46e2-82de-2c89ee74bed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:53.126506 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-95da5f9b-49c6-46e2-82de-2c89ee74bed6 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:53.128019 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-95da5f9b-49c6-46e2-82de-2c89ee74bed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:53.165649 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:53.192452 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1306/2598] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:58:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:58:53.200109 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b77950-97eb-425b-838e-83d160ab9e1d None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:53.200424 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b77950-97eb-425b-838e-83d160ab9e1d None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:53.200714 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b77950-97eb-425b-838e-83d160ab9e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:53.221738 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-d0b77950-97eb-425b-838e-83d160ab9e1d None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:53.223086 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-d0b77950-97eb-425b-838e-83d160ab9e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:53.231552 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1293/2599] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:58:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:58:53.235942 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95da5f9b-49c6-46e2-82de-2c89ee74bed6 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:53.246311 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95da5f9b-49c6-46e2-82de-2c89ee74bed6 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:53.250967 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e21b9-2889-47c1-8d07-00823d35251d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:53.251571 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e21b9-2889-47c1-8d07-00823d35251d admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:53.252392 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e21b9-2889-47c1-8d07-00823d35251d admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:53.252702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e21b9-2889-47c1-8d07-00823d35251d admin admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:53.253351 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e21b9-2889-47c1-8d07-00823d35251d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:53.253657 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e21b9-2889-47c1-8d07-00823d35251d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:53.255284 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e21b9-2889-47c1-8d07-00823d35251d admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:53.266947 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1307/2600] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:58:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:53.271875 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b77950-97eb-425b-838e-83d160ab9e1d None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:53.281360 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b77950-97eb-425b-838e-83d160ab9e1d None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:53.286245 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03ab94-6cd6-436e-92a2-e3c7c2f353af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:53.286842 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03ab94-6cd6-436e-92a2-e3c7c2f353af admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:53.287153 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03ab94-6cd6-436e-92a2-e3c7c2f353af admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:53.287425 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03ab94-6cd6-436e-92a2-e3c7c2f353af admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:53.288045 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a03ab94-6cd6-436e-92a2-e3c7c2f353af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:53.289981 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a03ab94-6cd6-436e-92a2-e3c7c2f353af admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:53.369870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1294/2601] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:58:53 2022] POST /identity/v3/projects => generated 381 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:58:53.373998 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e21b9-2889-47c1-8d07-00823d35251d admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:53.387312 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e21b9-2889-47c1-8d07-00823d35251d admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:53.457862 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8eca1b-5bb2-4df2-afaa-4a49fe66a509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:53.458410 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8eca1b-5bb2-4df2-afaa-4a49fe66a509 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:53.458863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8eca1b-5bb2-4df2-afaa-4a49fe66a509 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:53.459145 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8eca1b-5bb2-4df2-afaa-4a49fe66a509 admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:53.459787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8eca1b-5bb2-4df2-afaa-4a49fe66a509 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:53.461781 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8eca1b-5bb2-4df2-afaa-4a49fe66a509 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:53.563442 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1308/2602] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:58:53 2022] POST /identity/v3/users => generated 291 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:58:53.568235 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03ab94-6cd6-436e-92a2-e3c7c2f353af admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:53.578637 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03ab94-6cd6-436e-92a2-e3c7c2f353af admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:53.583821 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3682e5ca-5dd4-4809-884d-40ac679c4951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:53.584531 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3682e5ca-5dd4-4809-884d-40ac679c4951 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:53.584815 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3682e5ca-5dd4-4809-884d-40ac679c4951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:53.585093 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3682e5ca-5dd4-4809-884d-40ac679c4951 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:53.585735 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3682e5ca-5dd4-4809-884d-40ac679c4951 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:53.587407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3682e5ca-5dd4-4809-884d-40ac679c4951 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:53.597575 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1295/2603] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:58:53.601618 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8eca1b-5bb2-4df2-afaa-4a49fe66a509 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:53.612441 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8eca1b-5bb2-4df2-afaa-4a49fe66a509 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:53.617084 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bcbc2e-0cc2-4a65-92e9-0664f844a1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:53.617677 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bcbc2e-0cc2-4a65-92e9-0664f844a1ed admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:53.617954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bcbc2e-0cc2-4a65-92e9-0664f844a1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:53.618228 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bcbc2e-0cc2-4a65-92e9-0664f844a1ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:53.618876 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bcbc2e-0cc2-4a65-92e9-0664f844a1ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:53.620488 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bcbc2e-0cc2-4a65-92e9-0664f844a1ed admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:53.632972 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1309/2604] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:58:53.637292 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3682e5ca-5dd4-4809-884d-40ac679c4951 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:53.647585 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3682e5ca-5dd4-4809-884d-40ac679c4951 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:53.652705 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44713815-6638-4eb5-8146-47458de68789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:53.653355 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44713815-6638-4eb5-8146-47458de68789 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:53.653648 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44713815-6638-4eb5-8146-47458de68789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:53.653915 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44713815-6638-4eb5-8146-47458de68789 admin admin] PATH_INFO: `/v3/projects/4c6ba7a7881546eea41c15fa4c6e0244/users/89b5395d46bf4d55bbce8e30198eab6d/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:53.656103 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:53.670263 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44713815-6638-4eb5-8146-47458de68789 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c6ba7a7881546eea41c15fa4c6e0244, user_id=89b5395d46bf4d55bbce8e30198eab6d, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:53.672945 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44713815-6638-4eb5-8146-47458de68789 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:53.735908 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1296/2605] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:58:53 2022] PUT /identity/v3/projects/4c6ba7a7881546eea41c15fa4c6e0244/users/89b5395d46bf4d55bbce8e30198eab6d/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:58:53.741741 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15905a16-b9fe-46f2-a6e5-a8ea9db04cf4 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:53.744834 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15905a16-b9fe-46f2-a6e5-a8ea9db04cf4 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:53.744834 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15905a16-b9fe-46f2-a6e5-a8ea9db04cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:53.763417 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-15905a16-b9fe-46f2-a6e5-a8ea9db04cf4 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:53.767911 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-15905a16-b9fe-46f2-a6e5-a8ea9db04cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:53.828674 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1310/2606] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:58:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:58:53.837856 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44713815-6638-4eb5-8146-47458de68789 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:53.839608 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-44713815-6638-4eb5-8146-47458de68789 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:53.896590 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44713815-6638-4eb5-8146-47458de68789 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:53.900192 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f66d1c-a713-4f21-87de-5fdaf059e442 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:53.900903 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f66d1c-a713-4f21-87de-5fdaf059e442 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:53.901185 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f66d1c-a713-4f21-87de-5fdaf059e442 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:53.901458 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f66d1c-a713-4f21-87de-5fdaf059e442 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:53.911195 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f66d1c-a713-4f21-87de-5fdaf059e442 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:53.912660 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f66d1c-a713-4f21-87de-5fdaf059e442 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:53.923523 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1297/2607] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:58:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:58:55.621738 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9859386-99f9-4e54-bfbb-6204711d35fd None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:55.622301 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9859386-99f9-4e54-bfbb-6204711d35fd None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:55.622301 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9859386-99f9-4e54-bfbb-6204711d35fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:55.652911 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-d9859386-99f9-4e54-bfbb-6204711d35fd None None] MFA Rules not processed for user `89b5395d46bf4d55bbce8e30198eab6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:55.654615 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-d9859386-99f9-4e54-bfbb-6204711d35fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:55.691397 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:55.724136 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1311/2608] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:58:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:58:55.734707 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f66d1c-a713-4f21-87de-5fdaf059e442 service neutron] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:55.740800 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-a6f66d1c-a713-4f21-87de-5fdaf059e442 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:55.795538 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f66d1c-a713-4f21-87de-5fdaf059e442 service neutron] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:55.799784 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32abb165-ea35-43a6-87d8-c92ea6efe460 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:55.800843 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32abb165-ea35-43a6-87d8-c92ea6efe460 service cinder] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:55.800981 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32abb165-ea35-43a6-87d8-c92ea6efe460 service cinder] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:55.801213 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32abb165-ea35-43a6-87d8-c92ea6efe460 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:55.809512 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32abb165-ea35-43a6-87d8-c92ea6efe460 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:55.810938 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32abb165-ea35-43a6-87d8-c92ea6efe460 service cinder] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:55.819807 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1298/2609] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:58:55 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:58:58.707181 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2134d12-de9f-400c-9aae-a6c8246f3e49 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:58.707776 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2134d12-de9f-400c-9aae-a6c8246f3e49 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:58.707776 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2134d12-de9f-400c-9aae-a6c8246f3e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:58.728176 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-a2134d12-de9f-400c-9aae-a6c8246f3e49 None None] MFA Rules not processed for user `89b5395d46bf4d55bbce8e30198eab6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:58.729439 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-a2134d12-de9f-400c-9aae-a6c8246f3e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:58.737003 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1312/2610] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:58:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:58:58.983450 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32abb165-ea35-43a6-87d8-c92ea6efe460 service cinder] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:58.993544 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32abb165-ea35-43a6-87d8-c92ea6efe460 service cinder] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:58.997125 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b103b0-fce2-43e4-90fa-c985cd4b7977 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:58.997746 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b103b0-fce2-43e4-90fa-c985cd4b7977 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:58.998023 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b103b0-fce2-43e4-90fa-c985cd4b7977 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:58.998295 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b103b0-fce2-43e4-90fa-c985cd4b7977 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:59.000101 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-42b103b0-fce2-43e4-90fa-c985cd4b7977 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:59.011531 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b103b0-fce2-43e4-90fa-c985cd4b7977 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:59.013063 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b103b0-fce2-43e4-90fa-c985cd4b7977 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:59.022997 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1299/2611] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:58:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:59:08.723477 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=14 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:09.019237 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=65,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:11.912837 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2134d12-de9f-400c-9aae-a6c8246f3e49 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:11.914430 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-a2134d12-de9f-400c-9aae-a6c8246f3e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:12.005835 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2134d12-de9f-400c-9aae-a6c8246f3e49 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:12.011921 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427eddd2-c0ba-4d43-88c5-d311881d4987 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:12.012536 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427eddd2-c0ba-4d43-88c5-d311881d4987 service nova] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:12.013254 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427eddd2-c0ba-4d43-88c5-d311881d4987 service nova] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:12.013254 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427eddd2-c0ba-4d43-88c5-d311881d4987 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:12.014936 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-427eddd2-c0ba-4d43-88c5-d311881d4987 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:12.082691 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427eddd2-c0ba-4d43-88c5-d311881d4987 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:12.084308 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427eddd2-c0ba-4d43-88c5-d311881d4987 service nova] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:12.093877 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1313/2612] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:59:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:59:22.089422 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=56 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:25.467903 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b103b0-fce2-43e4-90fa-c985cd4b7977 service neutron] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:25.477415 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b103b0-fce2-43e4-90fa-c985cd4b7977 service neutron] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:25.481098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda3b02d-02d1-4ae2-bd1c-5a935a0ec2b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:25.481715 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3b02d-02d1-4ae2-bd1c-5a935a0ec2b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:25.482025 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3b02d-02d1-4ae2-bd1c-5a935a0ec2b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:25.482301 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3b02d-02d1-4ae2-bd1c-5a935a0ec2b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:25.484049 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-eda3b02d-02d1-4ae2-bd1c-5a935a0ec2b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:25.498478 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda3b02d-02d1-4ae2-bd1c-5a935a0ec2b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:25.499909 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda3b02d-02d1-4ae2-bd1c-5a935a0ec2b5 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:25.511957 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1300/2613] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:59:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:59:25.920276 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427eddd2-c0ba-4d43-88c5-d311881d4987 service nova] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:25.920276 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-427eddd2-c0ba-4d43-88c5-d311881d4987 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:25.936428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427eddd2-c0ba-4d43-88c5-d311881d4987 service nova] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:25.940447 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331fa33-fe95-42c9-ba5e-6efff7f6183a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:25.941431 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331fa33-fe95-42c9-ba5e-6efff7f6183a service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:25.941431 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331fa33-fe95-42c9-ba5e-6efff7f6183a service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:25.941690 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331fa33-fe95-42c9-ba5e-6efff7f6183a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:25.945480 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-6331fa33-fe95-42c9-ba5e-6efff7f6183a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:25.958802 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331fa33-fe95-42c9-ba5e-6efff7f6183a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:25.960287 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331fa33-fe95-42c9-ba5e-6efff7f6183a service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:25.971810 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1314/2614] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:59:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:59:26.715511 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda3b02d-02d1-4ae2-bd1c-5a935a0ec2b5 service neutron] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:26.725277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda3b02d-02d1-4ae2-bd1c-5a935a0ec2b5 service neutron] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:26.729224 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991ebd4-0d57-45b9-8b9c-2bc5d12070c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:26.730114 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991ebd4-0d57-45b9-8b9c-2bc5d12070c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:26.730567 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991ebd4-0d57-45b9-8b9c-2bc5d12070c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:26.733662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991ebd4-0d57-45b9-8b9c-2bc5d12070c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:26.733662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-7991ebd4-0d57-45b9-8b9c-2bc5d12070c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:26.749390 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7991ebd4-0d57-45b9-8b9c-2bc5d12070c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:26.750884 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7991ebd4-0d57-45b9-8b9c-2bc5d12070c5 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:26.763338 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1301/2615] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:59:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:59:29.377082 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331fa33-fe95-42c9-ba5e-6efff7f6183a service neutron] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:29.378532 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-6331fa33-fe95-42c9-ba5e-6efff7f6183a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:29.451136 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331fa33-fe95-42c9-ba5e-6efff7f6183a service neutron] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:29.459027 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b023936d-e35b-4cb5-b748-e180248e98ae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6778b7c5ad19484b877494f143050405', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:29.459649 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023936d-e35b-4cb5-b748-e180248e98ae service placement] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:29.459938 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023936d-e35b-4cb5-b748-e180248e98ae service placement] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:29.460213 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023936d-e35b-4cb5-b748-e180248e98ae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:29.462046 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-b023936d-e35b-4cb5-b748-e180248e98ae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:29.474181 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b023936d-e35b-4cb5-b748-e180248e98ae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:29.476001 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b023936d-e35b-4cb5-b748-e180248e98ae service placement] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:29.486269 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1315/2616] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:59:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:59:29.503193 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991ebd4-0d57-45b9-8b9c-2bc5d12070c5 service neutron] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:29.505656 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-7991ebd4-0d57-45b9-8b9c-2bc5d12070c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:29.528502 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991ebd4-0d57-45b9-8b9c-2bc5d12070c5 service neutron] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:29.533766 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c1736-87e8-4a33-bace-1e6db365b40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:29.534401 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c1736-87e8-4a33-bace-1e6db365b40a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:29.534683 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c1736-87e8-4a33-bace-1e6db365b40a admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:29.534946 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c1736-87e8-4a33-bace-1e6db365b40a admin admin] PATH_INFO: `/v3/users/c7ef3bf8dfd342f5a2fef02bf597af90` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:29.536078 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2c1736-87e8-4a33-bace-1e6db365b40a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7ef3bf8dfd342f5a2fef02bf597af90)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:29.538213 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2c1736-87e8-4a33-bace-1e6db365b40a admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:29.711295 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-aa2c1736-87e8-4a33-bace-1e6db365b40a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7ef3bf8dfd342f5a2fef02bf597af90'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:29.725706 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-aa2c1736-87e8-4a33-bace-1e6db365b40a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7ef3bf8dfd342f5a2fef02bf597af90'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:29.726726 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-aa2c1736-87e8-4a33-bace-1e6db365b40a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7ef3bf8dfd342f5a2fef02bf597af90'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:29.790925 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-aa2c1736-87e8-4a33-bace-1e6db365b40a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7ef3bf8dfd342f5a2fef02bf597af90'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:29.809027 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-aa2c1736-87e8-4a33-bace-1e6db365b40a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7ef3bf8dfd342f5a2fef02bf597af90'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:29.816348 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1302/2617] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:59:29 2022] DELETE /identity/v3/users/c7ef3bf8dfd342f5a2fef02bf597af90 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:59:30.078208 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b023936d-e35b-4cb5-b748-e180248e98ae service placement] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:30.098545 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b023936d-e35b-4cb5-b748-e180248e98ae service placement] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:30.173446 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b52e69-7bfe-435a-88c9-8fa7aa4f8381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:30.174446 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b52e69-7bfe-435a-88c9-8fa7aa4f8381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:30.174763 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b52e69-7bfe-435a-88c9-8fa7aa4f8381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:30.175052 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b52e69-7bfe-435a-88c9-8fa7aa4f8381 admin admin] PATH_INFO: `/v3/projects/56fbd96812654d16bbcf3522574a67eb` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:30.176526 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b52e69-7bfe-435a-88c9-8fa7aa4f8381 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56fbd96812654d16bbcf3522574a67eb)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:30.178560 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b52e69-7bfe-435a-88c9-8fa7aa4f8381 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:30.319623 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-39b52e69-7bfe-435a-88c9-8fa7aa4f8381 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56fbd96812654d16bbcf3522574a67eb', 'request_id': 'req-39b52e69-7bfe-435a-88c9-8fa7aa4f8381'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:30.325855 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-39b52e69-7bfe-435a-88c9-8fa7aa4f8381 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56fbd96812654d16bbcf3522574a67eb', 'request_id': 'req-39b52e69-7bfe-435a-88c9-8fa7aa4f8381'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:30.396603 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-39b52e69-7bfe-435a-88c9-8fa7aa4f8381 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56fbd96812654d16bbcf3522574a67eb', 'request_id': 'req-39b52e69-7bfe-435a-88c9-8fa7aa4f8381'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:30.409577 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-39b52e69-7bfe-435a-88c9-8fa7aa4f8381 admin admin] The token cache is being invalidate because project 56fbd96812654d16bbcf3522574a67eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:59:30.409993 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-39b52e69-7bfe-435a-88c9-8fa7aa4f8381 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:30.410736 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-39b52e69-7bfe-435a-88c9-8fa7aa4f8381 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:30.412366 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1316/2618] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:59:30 2022] DELETE /identity/v3/projects/56fbd96812654d16bbcf3522574a67eb => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:59:30.417460 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f43087-c34d-4114-89f6-37c1b3f57621 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:30.417845 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f43087-c34d-4114-89f6-37c1b3f57621 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:30.418192 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f43087-c34d-4114-89f6-37c1b3f57621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:30.438815 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-b9f43087-c34d-4114-89f6-37c1b3f57621 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:59:30.440124 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-b9f43087-c34d-4114-89f6-37c1b3f57621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:30.502112 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1303/2619] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:59:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:59:30.511270 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6f966-1e0b-4c8d-8937-bdb75afc58e6 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:30.511559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6f966-1e0b-4c8d-8937-bdb75afc58e6 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:30.511837 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6f966-1e0b-4c8d-8937-bdb75afc58e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:30.527265 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:30.533516 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-b4e6f966-1e0b-4c8d-8937-bdb75afc58e6 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:59:30.534709 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-b4e6f966-1e0b-4c8d-8937-bdb75afc58e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:30.542894 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1317/2620] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:59:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:59:30.547385 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f43087-c34d-4114-89f6-37c1b3f57621 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:30.557542 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f43087-c34d-4114-89f6-37c1b3f57621 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:30.562674 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72409c1-eda8-46ec-a581-f927183c494c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:30.563331 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72409c1-eda8-46ec-a581-f927183c494c admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:30.563611 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72409c1-eda8-46ec-a581-f927183c494c admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:30.563962 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72409c1-eda8-46ec-a581-f927183c494c admin admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:30.564644 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72409c1-eda8-46ec-a581-f927183c494c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:59:30.564954 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72409c1-eda8-46ec-a581-f927183c494c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:30.566626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72409c1-eda8-46ec-a581-f927183c494c admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:30.578247 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1304/2621] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:59:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:59:30.583246 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e6f966-1e0b-4c8d-8937-bdb75afc58e6 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:30.592493 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e6f966-1e0b-4c8d-8937-bdb75afc58e6 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:30.597933 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bb0652-0c98-4da2-b1fa-45d0907e03d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:30.598533 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bb0652-0c98-4da2-b1fa-45d0907e03d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:30.598822 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bb0652-0c98-4da2-b1fa-45d0907e03d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:30.599103 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bb0652-0c98-4da2-b1fa-45d0907e03d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:30.599732 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bb0652-0c98-4da2-b1fa-45d0907e03d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:30.601694 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bb0652-0c98-4da2-b1fa-45d0907e03d0 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:30.672119 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1318/2622] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:59:30 2022] POST /identity/v3/projects => generated 405 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:59:30.676763 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72409c1-eda8-46ec-a581-f927183c494c admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:30.686814 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72409c1-eda8-46ec-a581-f927183c494c admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:30.716627 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:30.744584 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb3b047-f7a2-4979-bf8b-843fbaa64964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:30.745194 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3b047-f7a2-4979-bf8b-843fbaa64964 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:30.745544 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3b047-f7a2-4979-bf8b-843fbaa64964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:30.745909 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3b047-f7a2-4979-bf8b-843fbaa64964 admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:30.746533 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb3b047-f7a2-4979-bf8b-843fbaa64964 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:30.748627 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb3b047-f7a2-4979-bf8b-843fbaa64964 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:30.819860 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1305/2623] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:59:30 2022] POST /identity/v3/users => generated 303 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:59:30.824226 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bb0652-0c98-4da2-b1fa-45d0907e03d0 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:30.835313 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bb0652-0c98-4da2-b1fa-45d0907e03d0 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:30.839900 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8660931d-7b12-4b1c-a5d0-b2d6cdacda8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:30.840540 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660931d-7b12-4b1c-a5d0-b2d6cdacda8e admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:30.840819 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660931d-7b12-4b1c-a5d0-b2d6cdacda8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:30.841096 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660931d-7b12-4b1c-a5d0-b2d6cdacda8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:30.841742 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8660931d-7b12-4b1c-a5d0-b2d6cdacda8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:30.843348 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8660931d-7b12-4b1c-a5d0-b2d6cdacda8e admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:30.853764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1319/2624] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:59:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:59:30.857745 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb3b047-f7a2-4979-bf8b-843fbaa64964 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:30.868151 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb3b047-f7a2-4979-bf8b-843fbaa64964 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:30.872742 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf18b0-7805-4743-af76-804efeede587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:30.873337 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf18b0-7805-4743-af76-804efeede587 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:30.873622 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf18b0-7805-4743-af76-804efeede587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:30.873894 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf18b0-7805-4743-af76-804efeede587 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:30.874529 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cf18b0-7805-4743-af76-804efeede587 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:30.876852 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cf18b0-7805-4743-af76-804efeede587 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:30.886690 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1306/2625] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:59:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:59:30.890793 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8660931d-7b12-4b1c-a5d0-b2d6cdacda8e admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:30.901310 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8660931d-7b12-4b1c-a5d0-b2d6cdacda8e admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:30.905941 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40642510-3f02-4b83-94be-e1fe339c58f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:30.906586 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40642510-3f02-4b83-94be-e1fe339c58f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:30.906867 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40642510-3f02-4b83-94be-e1fe339c58f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:30.907176 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40642510-3f02-4b83-94be-e1fe339c58f3 admin admin] PATH_INFO: `/v3/projects/bd20bf2bbab5470db44cb466a588374b/users/894ad1007a8d46fea7f212304e12a1e0/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:30.921461 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40642510-3f02-4b83-94be-e1fe339c58f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd20bf2bbab5470db44cb466a588374b, user_id=894ad1007a8d46fea7f212304e12a1e0, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:30.924189 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40642510-3f02-4b83-94be-e1fe339c58f3 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:31.021707 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1320/2626] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:59:30 2022] PUT /identity/v3/projects/bd20bf2bbab5470db44cb466a588374b/users/894ad1007a8d46fea7f212304e12a1e0/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:59:31.027267 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05fd29-9c0b-409f-ba08-4155e471a3ed None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:31.027557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05fd29-9c0b-409f-ba08-4155e471a3ed None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:31.027841 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05fd29-9c0b-409f-ba08-4155e471a3ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:31.049801 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-0b05fd29-9c0b-409f-ba08-4155e471a3ed None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:59:31.051364 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-0b05fd29-9c0b-409f-ba08-4155e471a3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:31.115646 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1307/2627] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:59:31.139137 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40642510-3f02-4b83-94be-e1fe339c58f3 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:31.139137 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-40642510-3f02-4b83-94be-e1fe339c58f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:31.200843 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40642510-3f02-4b83-94be-e1fe339c58f3 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:31.204791 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47106d0c-4222-41d0-9299-6c2551b27265 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:31.205403 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47106d0c-4222-41d0-9299-6c2551b27265 service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:31.205681 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47106d0c-4222-41d0-9299-6c2551b27265 service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:31.205954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47106d0c-4222-41d0-9299-6c2551b27265 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:31.215320 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47106d0c-4222-41d0-9299-6c2551b27265 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:31.216781 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47106d0c-4222-41d0-9299-6c2551b27265 service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:31.226109 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1321/2628] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:59:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:59:32.490922 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3398726-9182-40db-a199-94c9f183e8b7 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:32.491547 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3398726-9182-40db-a199-94c9f183e8b7 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:32.491547 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3398726-9182-40db-a199-94c9f183e8b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:32.523435 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-a3398726-9182-40db-a199-94c9f183e8b7 None None] MFA Rules not processed for user `894ad1007a8d46fea7f212304e12a1e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:59:32.524678 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-a3398726-9182-40db-a199-94c9f183e8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:32.593138 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1308/2629] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:59:32 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:59:32.598222 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47106d0c-4222-41d0-9299-6c2551b27265 service neutron] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:32.608319 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47106d0c-4222-41d0-9299-6c2551b27265 service neutron] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:32.614217 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40de464-b5c9-4b29-ae69-203c1c739824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:32.614762 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40de464-b5c9-4b29-ae69-203c1c739824 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:32.615047 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40de464-b5c9-4b29-ae69-203c1c739824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:32.615326 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40de464-b5c9-4b29-ae69-203c1c739824 admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:32.615968 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40de464-b5c9-4b29-ae69-203c1c739824 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:32.617927 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40de464-b5c9-4b29-ae69-203c1c739824 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:32.701707 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1322/2630] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:59:32 2022] POST /identity/v3/projects => generated 405 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:59:32.705849 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3398726-9182-40db-a199-94c9f183e8b7 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:32.717056 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3398726-9182-40db-a199-94c9f183e8b7 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:32.760998 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:32.777648 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc04bbb-6c6f-4e9b-9dac-f1463c5dfef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:32.778264 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc04bbb-6c6f-4e9b-9dac-f1463c5dfef3 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:32.778547 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc04bbb-6c6f-4e9b-9dac-f1463c5dfef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:32.778826 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc04bbb-6c6f-4e9b-9dac-f1463c5dfef3 admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:32.779464 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc04bbb-6c6f-4e9b-9dac-f1463c5dfef3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:32.781545 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc04bbb-6c6f-4e9b-9dac-f1463c5dfef3 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:32.857628 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1309/2631] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:59:32 2022] POST /identity/v3/users => generated 309 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:59:32.859155 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40de464-b5c9-4b29-ae69-203c1c739824 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:32.870926 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40de464-b5c9-4b29-ae69-203c1c739824 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:32.877220 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9368d607-2896-4c42-baf9-521045e383cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:32.877841 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9368d607-2896-4c42-baf9-521045e383cc admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:32.878174 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9368d607-2896-4c42-baf9-521045e383cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:32.878461 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9368d607-2896-4c42-baf9-521045e383cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:32.880252 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9368d607-2896-4c42-baf9-521045e383cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:32.880945 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9368d607-2896-4c42-baf9-521045e383cc admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:32.893008 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1323/2632] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:59:32.897270 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc04bbb-6c6f-4e9b-9dac-f1463c5dfef3 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:32.907821 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc04bbb-6c6f-4e9b-9dac-f1463c5dfef3 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:32.912648 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed1ea0e-e8ea-4b5a-ae5c-00847ad36a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:32.913305 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1ea0e-e8ea-4b5a-ae5c-00847ad36a5a admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:32.913606 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1ea0e-e8ea-4b5a-ae5c-00847ad36a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:32.914003 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1ea0e-e8ea-4b5a-ae5c-00847ad36a5a admin admin] PATH_INFO: `/v3/projects/8ac28e192537486eac86d13fc524c310/users/703b71e7a5a9489fb387ab46b06adada/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:32.933445 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed1ea0e-e8ea-4b5a-ae5c-00847ad36a5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ac28e192537486eac86d13fc524c310, user_id=703b71e7a5a9489fb387ab46b06adada, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:32.936118 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed1ea0e-e8ea-4b5a-ae5c-00847ad36a5a admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:33.002583 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1310/2633] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:59:32 2022] PUT /identity/v3/projects/8ac28e192537486eac86d13fc524c310/users/703b71e7a5a9489fb387ab46b06adada/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:59:33.006810 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9368d607-2896-4c42-baf9-521045e383cc admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:33.018190 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9368d607-2896-4c42-baf9-521045e383cc admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:33.076893 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1023ca-b735-445b-a5d8-685199ef1e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:33.077510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1023ca-b735-445b-a5d8-685199ef1e4b admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:33.077797 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1023ca-b735-445b-a5d8-685199ef1e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:33.078076 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1023ca-b735-445b-a5d8-685199ef1e4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:33.078719 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1023ca-b735-445b-a5d8-685199ef1e4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:33.080881 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1023ca-b735-445b-a5d8-685199ef1e4b admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:33.090575 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1324/2634] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:59:33 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:59:33.095522 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed1ea0e-e8ea-4b5a-ae5c-00847ad36a5a admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:33.105912 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed1ea0e-e8ea-4b5a-ae5c-00847ad36a5a admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:33.111238 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949ccbe-84ec-4a79-8da2-204e4449ec82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:33.111882 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949ccbe-84ec-4a79-8da2-204e4449ec82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:33.112162 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949ccbe-84ec-4a79-8da2-204e4449ec82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:33.112465 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949ccbe-84ec-4a79-8da2-204e4449ec82 admin admin] PATH_INFO: `/v3/projects/8ac28e192537486eac86d13fc524c310/users/703b71e7a5a9489fb387ab46b06adada/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:33.114377 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b949ccbe-84ec-4a79-8da2-204e4449ec82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ac28e192537486eac86d13fc524c310, user_id=703b71e7a5a9489fb387ab46b06adada, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:33.117072 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b949ccbe-84ec-4a79-8da2-204e4449ec82 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:33.127311 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1311/2635] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:59:33 2022] PUT /identity/v3/projects/8ac28e192537486eac86d13fc524c310/users/703b71e7a5a9489fb387ab46b06adada/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:59:34.271884 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e99f0d-2216-4478-9423-8a4157ab015b None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:34.272479 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e99f0d-2216-4478-9423-8a4157ab015b None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:34.272479 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e99f0d-2216-4478-9423-8a4157ab015b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:34.293807 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:34.303551 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-c7e99f0d-2216-4478-9423-8a4157ab015b None None] MFA Rules not processed for user `703b71e7a5a9489fb387ab46b06adada`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:59:34.304822 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-c7e99f0d-2216-4478-9423-8a4157ab015b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:34.379881 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1325/2636] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:59:34 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:59:34.913433 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949ccbe-84ec-4a79-8da2-204e4449ec82 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:34.915197 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-b949ccbe-84ec-4a79-8da2-204e4449ec82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:34.988022 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949ccbe-84ec-4a79-8da2-204e4449ec82 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:34.992968 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f538db-f403-4c4a-9802-4c1298d3a32e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6778b7c5ad19484b877494f143050405', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:34.993550 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f538db-f403-4c4a-9802-4c1298d3a32e service placement] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:34.993828 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f538db-f403-4c4a-9802-4c1298d3a32e service placement] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:34.994101 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f538db-f403-4c4a-9802-4c1298d3a32e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:34.995822 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-61f538db-f403-4c4a-9802-4c1298d3a32e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:35.008091 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f538db-f403-4c4a-9802-4c1298d3a32e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:35.009714 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f538db-f403-4c4a-9802-4c1298d3a32e service placement] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:35.019780 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1312/2637] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:59:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:59:35.747954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e99f0d-2216-4478-9423-8a4157ab015b None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:35.814816 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e99f0d-2216-4478-9423-8a4157ab015b None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:35.819081 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6ade8-e9e8-4943-9814-c9b1a9f6cbb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:35.819710 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6ade8-e9e8-4943-9814-c9b1a9f6cbb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:35.819988 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6ade8-e9e8-4943-9814-c9b1a9f6cbb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:35.820261 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6ade8-e9e8-4943-9814-c9b1a9f6cbb0 admin admin] PATH_INFO: `/v3/users/894ad1007a8d46fea7f212304e12a1e0` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:35.821401 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca6ade8-e9e8-4943-9814-c9b1a9f6cbb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=894ad1007a8d46fea7f212304e12a1e0)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:35.823309 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca6ade8-e9e8-4943-9814-c9b1a9f6cbb0 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:36.045253 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-aca6ade8-e9e8-4943-9814-c9b1a9f6cbb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '894ad1007a8d46fea7f212304e12a1e0'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:36.139304 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-aca6ade8-e9e8-4943-9814-c9b1a9f6cbb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '894ad1007a8d46fea7f212304e12a1e0'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:36.148275 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-aca6ade8-e9e8-4943-9814-c9b1a9f6cbb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '894ad1007a8d46fea7f212304e12a1e0'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:36.160786 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-aca6ade8-e9e8-4943-9814-c9b1a9f6cbb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '894ad1007a8d46fea7f212304e12a1e0'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:36.161491 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-aca6ade8-e9e8-4943-9814-c9b1a9f6cbb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '894ad1007a8d46fea7f212304e12a1e0'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:36.167417 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1326/2638] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:59:35 2022] DELETE /identity/v3/users/894ad1007a8d46fea7f212304e12a1e0 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:59:36.420913 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f538db-f403-4c4a-9802-4c1298d3a32e service placement] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:36.435070 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f538db-f403-4c4a-9802-4c1298d3a32e service placement] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:36.511704 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11235a7c-9590-4783-a132-bc6d973a08a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:36.512340 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11235a7c-9590-4783-a132-bc6d973a08a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:36.512656 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11235a7c-9590-4783-a132-bc6d973a08a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:36.512934 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11235a7c-9590-4783-a132-bc6d973a08a4 admin admin] PATH_INFO: `/v3/projects/bd20bf2bbab5470db44cb466a588374b` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:36.514073 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11235a7c-9590-4783-a132-bc6d973a08a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd20bf2bbab5470db44cb466a588374b)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:36.515961 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11235a7c-9590-4783-a132-bc6d973a08a4 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:36.609544 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:36.628084 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-11235a7c-9590-4783-a132-bc6d973a08a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd20bf2bbab5470db44cb466a588374b', 'request_id': 'req-11235a7c-9590-4783-a132-bc6d973a08a4'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:36.634799 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-11235a7c-9590-4783-a132-bc6d973a08a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd20bf2bbab5470db44cb466a588374b', 'request_id': 'req-11235a7c-9590-4783-a132-bc6d973a08a4'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:36.813943 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-11235a7c-9590-4783-a132-bc6d973a08a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd20bf2bbab5470db44cb466a588374b', 'request_id': 'req-11235a7c-9590-4783-a132-bc6d973a08a4'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:36.824894 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-11235a7c-9590-4783-a132-bc6d973a08a4 admin admin] The token cache is being invalidate because project bd20bf2bbab5470db44cb466a588374b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:59:36.825189 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-11235a7c-9590-4783-a132-bc6d973a08a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:36.825861 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-11235a7c-9590-4783-a132-bc6d973a08a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:36.827364 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1313/2639] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:59:36 2022] DELETE /identity/v3/projects/bd20bf2bbab5470db44cb466a588374b => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:59:36.831613 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6ade8-e9e8-4943-9814-c9b1a9f6cbb0 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:36.833085 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-aca6ade8-e9e8-4943-9814-c9b1a9f6cbb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:36.847646 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6ade8-e9e8-4943-9814-c9b1a9f6cbb0 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:36.856454 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:36.904124 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c9b7e9-2f40-4fd1-ba6e-4af86d541545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:36.904826 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9b7e9-2f40-4fd1-ba6e-4af86d541545 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:36.905110 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9b7e9-2f40-4fd1-ba6e-4af86d541545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:36.905387 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9b7e9-2f40-4fd1-ba6e-4af86d541545 admin admin] PATH_INFO: `/v3/users/703b71e7a5a9489fb387ab46b06adada` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:36.906530 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c9b7e9-2f40-4fd1-ba6e-4af86d541545 admin admin] RBAC: Authorizing `identity:delete_user(user_id=703b71e7a5a9489fb387ab46b06adada)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:36.909171 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c9b7e9-2f40-4fd1-ba6e-4af86d541545 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:37.234414 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-71c9b7e9-2f40-4fd1-ba6e-4af86d541545 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '703b71e7a5a9489fb387ab46b06adada'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:37.310839 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-71c9b7e9-2f40-4fd1-ba6e-4af86d541545 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '703b71e7a5a9489fb387ab46b06adada'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:37.324668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-71c9b7e9-2f40-4fd1-ba6e-4af86d541545 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '703b71e7a5a9489fb387ab46b06adada'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:37.337662 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-71c9b7e9-2f40-4fd1-ba6e-4af86d541545 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '703b71e7a5a9489fb387ab46b06adada'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:37.338334 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-71c9b7e9-2f40-4fd1-ba6e-4af86d541545 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '703b71e7a5a9489fb387ab46b06adada'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:37.345262 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1327/2640] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:59:36 2022] DELETE /identity/v3/users/703b71e7a5a9489fb387ab46b06adada => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:59:37.598473 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11235a7c-9590-4783-a132-bc6d973a08a4 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:37.599937 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-11235a7c-9590-4783-a132-bc6d973a08a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:37.614076 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11235a7c-9590-4783-a132-bc6d973a08a4 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:37.690180 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d01fd4d-646b-4d1f-9fce-d2f6328f5d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:37.690982 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01fd4d-646b-4d1f-9fce-d2f6328f5d9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:37.691251 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01fd4d-646b-4d1f-9fce-d2f6328f5d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:37.691521 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01fd4d-646b-4d1f-9fce-d2f6328f5d9b admin admin] PATH_INFO: `/v3/projects/8ac28e192537486eac86d13fc524c310` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:37.692670 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d01fd4d-646b-4d1f-9fce-d2f6328f5d9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ac28e192537486eac86d13fc524c310)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:37.694559 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d01fd4d-646b-4d1f-9fce-d2f6328f5d9b admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:37.802929 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-7d01fd4d-646b-4d1f-9fce-d2f6328f5d9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ac28e192537486eac86d13fc524c310', 'request_id': 'req-7d01fd4d-646b-4d1f-9fce-d2f6328f5d9b'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:37.809043 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-7d01fd4d-646b-4d1f-9fce-d2f6328f5d9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ac28e192537486eac86d13fc524c310', 'request_id': 'req-7d01fd4d-646b-4d1f-9fce-d2f6328f5d9b'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:37.870846 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-7d01fd4d-646b-4d1f-9fce-d2f6328f5d9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ac28e192537486eac86d13fc524c310', 'request_id': 'req-7d01fd4d-646b-4d1f-9fce-d2f6328f5d9b'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:37.884131 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-7d01fd4d-646b-4d1f-9fce-d2f6328f5d9b admin admin] The token cache is being invalidate because project 8ac28e192537486eac86d13fc524c310 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:59:37.889052 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-7d01fd4d-646b-4d1f-9fce-d2f6328f5d9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:37.889052 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-7d01fd4d-646b-4d1f-9fce-d2f6328f5d9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:37.889052 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1314/2641] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:59:37 2022] DELETE /identity/v3/projects/8ac28e192537486eac86d13fc524c310 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:59:37.891716 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fddad8f-6b0a-45e3-9e80-9966a3905ccf None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:37.892212 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fddad8f-6b0a-45e3-9e80-9966a3905ccf None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:37.892337 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fddad8f-6b0a-45e3-9e80-9966a3905ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:37.911780 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-2fddad8f-6b0a-45e3-9e80-9966a3905ccf None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:59:37.913019 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-2fddad8f-6b0a-45e3-9e80-9966a3905ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:37.976892 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1328/2642] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:59:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:59:37.986308 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5077c4c3-c0f6-46f4-b0c3-e20af48687d6 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:37.986663 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5077c4c3-c0f6-46f4-b0c3-e20af48687d6 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:37.987012 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5077c4c3-c0f6-46f4-b0c3-e20af48687d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:38.007676 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-5077c4c3-c0f6-46f4-b0c3-e20af48687d6 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:59:38.008816 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-5077c4c3-c0f6-46f4-b0c3-e20af48687d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:38.016873 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1315/2643] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:59:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:59:38.020985 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fddad8f-6b0a-45e3-9e80-9966a3905ccf None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:38.030784 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fddad8f-6b0a-45e3-9e80-9966a3905ccf None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:38.035578 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995934b6-93af-4c88-b300-bc6be8f0db95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:38.036185 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995934b6-93af-4c88-b300-bc6be8f0db95 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:38.036513 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995934b6-93af-4c88-b300-bc6be8f0db95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:38.036795 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995934b6-93af-4c88-b300-bc6be8f0db95 admin admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:38.037444 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995934b6-93af-4c88-b300-bc6be8f0db95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:59:38.037751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995934b6-93af-4c88-b300-bc6be8f0db95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:38.039356 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995934b6-93af-4c88-b300-bc6be8f0db95 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:38.051825 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1329/2644] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:59:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:59:38.056179 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5077c4c3-c0f6-46f4-b0c3-e20af48687d6 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:38.066457 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5077c4c3-c0f6-46f4-b0c3-e20af48687d6 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:38.071568 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf94a915-c61d-457c-b1d2-d3bb7941b8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:38.071811 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94a915-c61d-457c-b1d2-d3bb7941b8cb admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:38.072094 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94a915-c61d-457c-b1d2-d3bb7941b8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:38.072371 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94a915-c61d-457c-b1d2-d3bb7941b8cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:38.073042 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf94a915-c61d-457c-b1d2-d3bb7941b8cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:38.074937 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf94a915-c61d-457c-b1d2-d3bb7941b8cb admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:38.215276 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1316/2645] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:59:38 2022] POST /identity/v3/projects => generated 365 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:59:38.220458 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995934b6-93af-4c88-b300-bc6be8f0db95 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:38.230694 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995934b6-93af-4c88-b300-bc6be8f0db95 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:38.248269 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:38.287048 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b8113-9d18-4608-b6d2-bb25bb59fb40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:38.287668 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b8113-9d18-4608-b6d2-bb25bb59fb40 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:38.287957 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b8113-9d18-4608-b6d2-bb25bb59fb40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:38.288233 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b8113-9d18-4608-b6d2-bb25bb59fb40 admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:38.288895 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8b8113-9d18-4608-b6d2-bb25bb59fb40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:38.290864 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8b8113-9d18-4608-b6d2-bb25bb59fb40 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:38.373883 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1330/2646] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:59:38 2022] POST /identity/v3/users => generated 283 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:59:38.378285 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf94a915-c61d-457c-b1d2-d3bb7941b8cb admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:38.388985 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf94a915-c61d-457c-b1d2-d3bb7941b8cb admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:38.393665 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9bcf6-ad5f-4d23-8ef9-39967fd465eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:38.394275 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9bcf6-ad5f-4d23-8ef9-39967fd465eb admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:38.394556 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9bcf6-ad5f-4d23-8ef9-39967fd465eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:38.394836 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9bcf6-ad5f-4d23-8ef9-39967fd465eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:38.395514 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e9bcf6-ad5f-4d23-8ef9-39967fd465eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:38.397143 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e9bcf6-ad5f-4d23-8ef9-39967fd465eb admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:38.407856 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1317/2647] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:59:38.417950 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b8113-9d18-4608-b6d2-bb25bb59fb40 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:38.429631 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b8113-9d18-4608-b6d2-bb25bb59fb40 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:38.435088 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a27ddd7-9460-4ad9-b2cf-db0a86278be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:38.435719 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a27ddd7-9460-4ad9-b2cf-db0a86278be5 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:38.436001 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a27ddd7-9460-4ad9-b2cf-db0a86278be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:38.436276 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a27ddd7-9460-4ad9-b2cf-db0a86278be5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:38.442555 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a27ddd7-9460-4ad9-b2cf-db0a86278be5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:38.442555 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a27ddd7-9460-4ad9-b2cf-db0a86278be5 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:38.449030 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1331/2648] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:59:38.453335 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9bcf6-ad5f-4d23-8ef9-39967fd465eb admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:38.459994 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:38.463778 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9bcf6-ad5f-4d23-8ef9-39967fd465eb admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:38.468515 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af00a2f-2487-41cf-b3e9-5497d468df20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:38.471438 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af00a2f-2487-41cf-b3e9-5497d468df20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:38.471438 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af00a2f-2487-41cf-b3e9-5497d468df20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:38.471438 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af00a2f-2487-41cf-b3e9-5497d468df20 admin admin] PATH_INFO: `/v3/projects/1dff50c3c89249179831b3998925b528/users/19f04e81ef8342f39f75f3a60d11e89f/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:38.485881 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af00a2f-2487-41cf-b3e9-5497d468df20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dff50c3c89249179831b3998925b528, user_id=19f04e81ef8342f39f75f3a60d11e89f, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:38.488880 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af00a2f-2487-41cf-b3e9-5497d468df20 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:38.556566 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1318/2649] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:59:38 2022] PUT /identity/v3/projects/1dff50c3c89249179831b3998925b528/users/19f04e81ef8342f39f75f3a60d11e89f/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:59:38.567510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ddff3b-d7a2-4b1c-b93d-0493cfedba19 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:38.567801 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ddff3b-d7a2-4b1c-b93d-0493cfedba19 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:38.568080 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ddff3b-d7a2-4b1c-b93d-0493cfedba19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:38.596948 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-97ddff3b-d7a2-4b1c-b93d-0493cfedba19 None None] MFA Rules not processed for user `19f04e81ef8342f39f75f3a60d11e89f`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:59:38.598151 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-97ddff3b-d7a2-4b1c-b93d-0493cfedba19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:38.662566 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1332/2650] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:59:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:59:38.667001 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af00a2f-2487-41cf-b3e9-5497d468df20 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:38.733484 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af00a2f-2487-41cf-b3e9-5497d468df20 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:38.738819 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e4b985-1ec3-4da7-83e7-b2c806af2891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:38.739282 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4b985-1ec3-4da7-83e7-b2c806af2891 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:38.739635 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4b985-1ec3-4da7-83e7-b2c806af2891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:38.742790 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4b985-1ec3-4da7-83e7-b2c806af2891 admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:38.742790 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e4b985-1ec3-4da7-83e7-b2c806af2891 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:38.742790 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e4b985-1ec3-4da7-83e7-b2c806af2891 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:38.855415 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1319/2651] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:59:38 2022] POST /identity/v3/projects => generated 365 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:59:38.859342 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ddff3b-d7a2-4b1c-b93d-0493cfedba19 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:38.870216 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ddff3b-d7a2-4b1c-b93d-0493cfedba19 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:38.929804 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea1c1f-1be4-4744-8cdd-ae0dac4c4df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:38.930499 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea1c1f-1be4-4744-8cdd-ae0dac4c4df5 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:38.930628 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea1c1f-1be4-4744-8cdd-ae0dac4c4df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:38.931419 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea1c1f-1be4-4744-8cdd-ae0dac4c4df5 admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:38.931419 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea1c1f-1be4-4744-8cdd-ae0dac4c4df5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:38.933374 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea1c1f-1be4-4744-8cdd-ae0dac4c4df5 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:39.037137 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1333/2652] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:59:38 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:59:39.041202 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e4b985-1ec3-4da7-83e7-b2c806af2891 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:39.051870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e4b985-1ec3-4da7-83e7-b2c806af2891 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:39.058117 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bede5f-9d60-4e2a-b54f-a720ce5069e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:39.058878 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bede5f-9d60-4e2a-b54f-a720ce5069e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:39.059259 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bede5f-9d60-4e2a-b54f-a720ce5069e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:39.059538 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bede5f-9d60-4e2a-b54f-a720ce5069e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:39.060208 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bede5f-9d60-4e2a-b54f-a720ce5069e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:39.061956 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bede5f-9d60-4e2a-b54f-a720ce5069e9 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:39.074966 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1320/2653] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:59:39.080073 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea1c1f-1be4-4744-8cdd-ae0dac4c4df5 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:39.090345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea1c1f-1be4-4744-8cdd-ae0dac4c4df5 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:39.095463 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b493f51-e827-41d8-b30e-694bd58f6677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:39.096121 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b493f51-e827-41d8-b30e-694bd58f6677 admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:39.096430 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b493f51-e827-41d8-b30e-694bd58f6677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:39.096710 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b493f51-e827-41d8-b30e-694bd58f6677 admin admin] PATH_INFO: `/v3/projects/087ab969717245568da24270faf9ad81/users/2ceebb83fae84532a32f07d48b5d1842/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:39.111741 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b493f51-e827-41d8-b30e-694bd58f6677 admin admin] RBAC: Authorizing `identity:create_grant(project_id=087ab969717245568da24270faf9ad81, user_id=2ceebb83fae84532a32f07d48b5d1842, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:39.114468 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b493f51-e827-41d8-b30e-694bd58f6677 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:39.116756 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:39.208865 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1334/2654] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:59:39 2022] PUT /identity/v3/projects/087ab969717245568da24270faf9ad81/users/2ceebb83fae84532a32f07d48b5d1842/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:59:39.213033 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bede5f-9d60-4e2a-b54f-a720ce5069e9 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:39.223069 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bede5f-9d60-4e2a-b54f-a720ce5069e9 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:39.278576 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a3871-f427-4fb2-b3b8-63f51bddf7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:39.279180 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a3871-f427-4fb2-b3b8-63f51bddf7e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:39.279491 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a3871-f427-4fb2-b3b8-63f51bddf7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:39.279772 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a3871-f427-4fb2-b3b8-63f51bddf7e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:39.280452 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920a3871-f427-4fb2-b3b8-63f51bddf7e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:39.282462 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920a3871-f427-4fb2-b3b8-63f51bddf7e8 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:39.292107 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1321/2655] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:59:39.296168 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b493f51-e827-41d8-b30e-694bd58f6677 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:39.308201 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b493f51-e827-41d8-b30e-694bd58f6677 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:39.313897 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204856d5-d6ef-4b76-8e58-ffe0d3b0b97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:39.314779 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204856d5-d6ef-4b76-8e58-ffe0d3b0b97a admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:39.315065 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204856d5-d6ef-4b76-8e58-ffe0d3b0b97a admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:39.315345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204856d5-d6ef-4b76-8e58-ffe0d3b0b97a admin admin] PATH_INFO: `/v3/projects/087ab969717245568da24270faf9ad81/users/2ceebb83fae84532a32f07d48b5d1842/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:39.317323 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204856d5-d6ef-4b76-8e58-ffe0d3b0b97a admin admin] RBAC: Authorizing `identity:create_grant(project_id=087ab969717245568da24270faf9ad81, user_id=2ceebb83fae84532a32f07d48b5d1842, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:39.320023 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204856d5-d6ef-4b76-8e58-ffe0d3b0b97a admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:39.330265 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1335/2656] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:59:39 2022] PUT /identity/v3/projects/087ab969717245568da24270faf9ad81/users/2ceebb83fae84532a32f07d48b5d1842/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:59:39.341343 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa0d98-7f1b-4498-8a66-08fd1b7c2c03 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:39.341633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa0d98-7f1b-4498-8a66-08fd1b7c2c03 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:39.341933 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa0d98-7f1b-4498-8a66-08fd1b7c2c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:39.369685 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-91fa0d98-7f1b-4498-8a66-08fd1b7c2c03 None None] MFA Rules not processed for user `2ceebb83fae84532a32f07d48b5d1842`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:59:39.370895 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-91fa0d98-7f1b-4498-8a66-08fd1b7c2c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:39.411679 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:39.453907 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1322/2657] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:59:39 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:59:39.465652 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204856d5-d6ef-4b76-8e58-ffe0d3b0b97a admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:39.467437 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-204856d5-d6ef-4b76-8e58-ffe0d3b0b97a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:39.529652 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204856d5-d6ef-4b76-8e58-ffe0d3b0b97a admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:39.533788 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4ea81-5628-45ca-bb4a-1b872ec9f6b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:39.534402 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4ea81-5628-45ca-bb4a-1b872ec9f6b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:39.534685 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4ea81-5628-45ca-bb4a-1b872ec9f6b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:39.534961 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4ea81-5628-45ca-bb4a-1b872ec9f6b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:39.544081 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb4ea81-5628-45ca-bb4a-1b872ec9f6b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:39.545501 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb4ea81-5628-45ca-bb4a-1b872ec9f6b4 service cinder] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:39.556086 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1336/2658] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:59:39 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:59:39.799653 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa0d98-7f1b-4498-8a66-08fd1b7c2c03 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:39.801766 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-91fa0d98-7f1b-4498-8a66-08fd1b7c2c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:39.815188 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa0d98-7f1b-4498-8a66-08fd1b7c2c03 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:39.819031 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18a3e12-1057-4ef5-b262-89d20ea13ae5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:39.819631 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a3e12-1057-4ef5-b262-89d20ea13ae5 service cinder] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:39.819907 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a3e12-1057-4ef5-b262-89d20ea13ae5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:39.820176 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a3e12-1057-4ef5-b262-89d20ea13ae5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:39.884319 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18a3e12-1057-4ef5-b262-89d20ea13ae5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:39.885844 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18a3e12-1057-4ef5-b262-89d20ea13ae5 service cinder] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:39.896264 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1323/2659] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:59:39 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:59:43.585056 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4ea81-5628-45ca-bb4a-1b872ec9f6b4 service cinder] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:43.653151 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4ea81-5628-45ca-bb4a-1b872ec9f6b4 service cinder] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:43.658520 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deaa290-a822-4440-ad13-cf7ea035e3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:43.659166 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deaa290-a822-4440-ad13-cf7ea035e3e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:43.659449 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deaa290-a822-4440-ad13-cf7ea035e3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:43.659726 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deaa290-a822-4440-ad13-cf7ea035e3e4 admin admin] PATH_INFO: `/v3/users/19f04e81ef8342f39f75f3a60d11e89f` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:43.660898 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deaa290-a822-4440-ad13-cf7ea035e3e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19f04e81ef8342f39f75f3a60d11e89f)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:43.662887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deaa290-a822-4440-ad13-cf7ea035e3e4 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:43.951190 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-9deaa290-a822-4440-ad13-cf7ea035e3e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19f04e81ef8342f39f75f3a60d11e89f'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:44.056812 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-9deaa290-a822-4440-ad13-cf7ea035e3e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19f04e81ef8342f39f75f3a60d11e89f'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:44.066577 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-9deaa290-a822-4440-ad13-cf7ea035e3e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19f04e81ef8342f39f75f3a60d11e89f'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:44.080265 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-9deaa290-a822-4440-ad13-cf7ea035e3e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19f04e81ef8342f39f75f3a60d11e89f'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:44.084899 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-9deaa290-a822-4440-ad13-cf7ea035e3e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19f04e81ef8342f39f75f3a60d11e89f'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:44.088639 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1337/2660] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:59:43 2022] DELETE /identity/v3/users/19f04e81ef8342f39f75f3a60d11e89f => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:59:44.094898 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0e4bc-563a-4bab-a556-92c4ea5da3af None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:44.094898 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0e4bc-563a-4bab-a556-92c4ea5da3af None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:44.095681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0e4bc-563a-4bab-a556-92c4ea5da3af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:44.119861 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-aee0e4bc-563a-4bab-a556-92c4ea5da3af None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:59:44.121474 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-aee0e4bc-563a-4bab-a556-92c4ea5da3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:44.203192 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1324/2661] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:59:44.210371 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deaa290-a822-4440-ad13-cf7ea035e3e4 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:44.211870 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-9deaa290-a822-4440-ad13-cf7ea035e3e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:44.217642 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:44.278533 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deaa290-a822-4440-ad13-cf7ea035e3e4 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:44.282880 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0202cc1-f314-476a-9ad8-cd600f3d0166 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:44.283519 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0202cc1-f314-476a-9ad8-cd600f3d0166 service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:44.283804 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0202cc1-f314-476a-9ad8-cd600f3d0166 service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:44.284083 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0202cc1-f314-476a-9ad8-cd600f3d0166 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:44.294452 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0202cc1-f314-476a-9ad8-cd600f3d0166 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:44.296098 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0202cc1-f314-476a-9ad8-cd600f3d0166 service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:44.307938 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1338/2662] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:59:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:59:44.861031 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee0e4bc-563a-4bab-a556-92c4ea5da3af None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:44.862883 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-aee0e4bc-563a-4bab-a556-92c4ea5da3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:44.879631 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee0e4bc-563a-4bab-a556-92c4ea5da3af None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:44.884316 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f230b-016c-4dcd-8716-01f605b179a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:44.885001 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f230b-016c-4dcd-8716-01f605b179a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:44.885284 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f230b-016c-4dcd-8716-01f605b179a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:44.885564 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f230b-016c-4dcd-8716-01f605b179a8 admin admin] PATH_INFO: `/v3/projects/1dff50c3c89249179831b3998925b528` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:44.887915 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f230b-016c-4dcd-8716-01f605b179a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dff50c3c89249179831b3998925b528)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:44.889891 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f230b-016c-4dcd-8716-01f605b179a8 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:45.009495 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-ea7f230b-016c-4dcd-8716-01f605b179a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dff50c3c89249179831b3998925b528', 'request_id': 'req-ea7f230b-016c-4dcd-8716-01f605b179a8'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:45.015712 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-ea7f230b-016c-4dcd-8716-01f605b179a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dff50c3c89249179831b3998925b528', 'request_id': 'req-ea7f230b-016c-4dcd-8716-01f605b179a8'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:45.100051 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-ea7f230b-016c-4dcd-8716-01f605b179a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dff50c3c89249179831b3998925b528', 'request_id': 'req-ea7f230b-016c-4dcd-8716-01f605b179a8'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:45.112021 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-ea7f230b-016c-4dcd-8716-01f605b179a8 admin admin] The token cache is being invalidate because project 1dff50c3c89249179831b3998925b528 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:59:45.112390 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-ea7f230b-016c-4dcd-8716-01f605b179a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:45.113056 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:45.113982 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-ea7f230b-016c-4dcd-8716-01f605b179a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:45.115800 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1325/2663] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:59:44 2022] DELETE /identity/v3/projects/1dff50c3c89249179831b3998925b528 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:59:45.119564 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0202cc1-f314-476a-9ad8-cd600f3d0166 service neutron] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:45.131752 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0202cc1-f314-476a-9ad8-cd600f3d0166 service neutron] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:45.206503 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb0cd6-f056-4043-bea6-b5f8ba27f483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:45.207405 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb0cd6-f056-4043-bea6-b5f8ba27f483 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:45.207877 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb0cd6-f056-4043-bea6-b5f8ba27f483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:45.208384 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb0cd6-f056-4043-bea6-b5f8ba27f483 admin admin] PATH_INFO: `/v3/users/2ceebb83fae84532a32f07d48b5d1842` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:45.209909 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cb0cd6-f056-4043-bea6-b5f8ba27f483 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ceebb83fae84532a32f07d48b5d1842)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:45.212017 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cb0cd6-f056-4043-bea6-b5f8ba27f483 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:45.439687 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-85cb0cd6-f056-4043-bea6-b5f8ba27f483 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ceebb83fae84532a32f07d48b5d1842'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:45.562084 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-85cb0cd6-f056-4043-bea6-b5f8ba27f483 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ceebb83fae84532a32f07d48b5d1842'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:45.571403 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-85cb0cd6-f056-4043-bea6-b5f8ba27f483 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ceebb83fae84532a32f07d48b5d1842'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:45.586148 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-85cb0cd6-f056-4043-bea6-b5f8ba27f483 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ceebb83fae84532a32f07d48b5d1842'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:45.586904 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-85cb0cd6-f056-4043-bea6-b5f8ba27f483 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ceebb83fae84532a32f07d48b5d1842'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:45.596487 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1339/2664] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:59:45 2022] DELETE /identity/v3/users/2ceebb83fae84532a32f07d48b5d1842 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:59:46.095729 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f230b-016c-4dcd-8716-01f605b179a8 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:46.097535 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-ea7f230b-016c-4dcd-8716-01f605b179a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:46.111081 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f230b-016c-4dcd-8716-01f605b179a8 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:46.175647 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3aaa3-1f89-45e9-9502-b1fffc623d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:46.176590 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3aaa3-1f89-45e9-9502-b1fffc623d00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:46.176952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3aaa3-1f89-45e9-9502-b1fffc623d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:46.177230 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3aaa3-1f89-45e9-9502-b1fffc623d00 admin admin] PATH_INFO: `/v3/projects/087ab969717245568da24270faf9ad81` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:46.178413 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f3aaa3-1f89-45e9-9502-b1fffc623d00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=087ab969717245568da24270faf9ad81)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:46.180305 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f3aaa3-1f89-45e9-9502-b1fffc623d00 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:46.322789 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-57f3aaa3-1f89-45e9-9502-b1fffc623d00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '087ab969717245568da24270faf9ad81', 'request_id': 'req-57f3aaa3-1f89-45e9-9502-b1fffc623d00'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:46.329546 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-57f3aaa3-1f89-45e9-9502-b1fffc623d00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '087ab969717245568da24270faf9ad81', 'request_id': 'req-57f3aaa3-1f89-45e9-9502-b1fffc623d00'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:46.402569 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-57f3aaa3-1f89-45e9-9502-b1fffc623d00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '087ab969717245568da24270faf9ad81', 'request_id': 'req-57f3aaa3-1f89-45e9-9502-b1fffc623d00'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:46.417288 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-57f3aaa3-1f89-45e9-9502-b1fffc623d00 admin admin] The token cache is being invalidate because project 087ab969717245568da24270faf9ad81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:59:46.417762 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-57f3aaa3-1f89-45e9-9502-b1fffc623d00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:46.418666 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-57f3aaa3-1f89-45e9-9502-b1fffc623d00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:46.420602 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1326/2665] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:59:46 2022] DELETE /identity/v3/projects/087ab969717245568da24270faf9ad81 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:59:46.425528 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f9b22-1832-4efe-adb9-348e72d851e5 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:46.426394 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f9b22-1832-4efe-adb9-348e72d851e5 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:46.426394 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f9b22-1832-4efe-adb9-348e72d851e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:46.447180 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-f88f9b22-1832-4efe-adb9-348e72d851e5 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:59:46.455245 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-f88f9b22-1832-4efe-adb9-348e72d851e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:46.467678 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:46.508436 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1340/2666] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:59:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:59:46.516783 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283ae0a-8487-43bb-807b-e24a3d1a96f7 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:46.517202 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283ae0a-8487-43bb-807b-e24a3d1a96f7 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:46.517575 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283ae0a-8487-43bb-807b-e24a3d1a96f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:46.537952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-1283ae0a-8487-43bb-807b-e24a3d1a96f7 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:59:46.539158 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-1283ae0a-8487-43bb-807b-e24a3d1a96f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:46.547239 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1327/2667] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:59:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:59:46.551790 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f9b22-1832-4efe-adb9-348e72d851e5 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:46.561603 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f9b22-1832-4efe-adb9-348e72d851e5 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:46.566953 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f5235c-69fe-49e3-b6ce-82ea31577530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:46.567566 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5235c-69fe-49e3-b6ce-82ea31577530 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:46.567853 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5235c-69fe-49e3-b6ce-82ea31577530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:46.568126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5235c-69fe-49e3-b6ce-82ea31577530 admin admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:46.568832 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f5235c-69fe-49e3-b6ce-82ea31577530 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:59:46.569125 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f5235c-69fe-49e3-b6ce-82ea31577530 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:46.570748 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f5235c-69fe-49e3-b6ce-82ea31577530 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:46.584043 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1341/2668] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:59:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:59:46.588795 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1283ae0a-8487-43bb-807b-e24a3d1a96f7 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:46.598380 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1283ae0a-8487-43bb-807b-e24a3d1a96f7 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:46.603178 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54494ad2-689c-425f-a992-14a2c2a7da7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:46.603786 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54494ad2-689c-425f-a992-14a2c2a7da7e admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:46.604071 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54494ad2-689c-425f-a992-14a2c2a7da7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:46.604348 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54494ad2-689c-425f-a992-14a2c2a7da7e admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:46.605014 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54494ad2-689c-425f-a992-14a2c2a7da7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:46.606934 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54494ad2-689c-425f-a992-14a2c2a7da7e admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:46.729278 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1328/2669] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:59:46 2022] POST /identity/v3/projects => generated 401 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:59:46.733894 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f5235c-69fe-49e3-b6ce-82ea31577530 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:46.744512 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f5235c-69fe-49e3-b6ce-82ea31577530 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:46.803684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d31d45-e6bb-4c5c-9a3f-9db4bd3b4881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:46.804216 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d31d45-e6bb-4c5c-9a3f-9db4bd3b4881 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:46.804520 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d31d45-e6bb-4c5c-9a3f-9db4bd3b4881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:46.804794 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d31d45-e6bb-4c5c-9a3f-9db4bd3b4881 admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:46.805414 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d31d45-e6bb-4c5c-9a3f-9db4bd3b4881 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:46.807346 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d31d45-e6bb-4c5c-9a3f-9db4bd3b4881 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:46.940582 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1342/2670] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:59:46 2022] POST /identity/v3/users => generated 301 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:59:46.944913 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54494ad2-689c-425f-a992-14a2c2a7da7e admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:46.960077 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54494ad2-689c-425f-a992-14a2c2a7da7e admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:46.965456 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b723700-41ac-4b90-8ed5-b37e196a297b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:46.966107 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b723700-41ac-4b90-8ed5-b37e196a297b admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:46.966384 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b723700-41ac-4b90-8ed5-b37e196a297b admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:46.966660 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b723700-41ac-4b90-8ed5-b37e196a297b admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:46.967308 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b723700-41ac-4b90-8ed5-b37e196a297b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:46.968981 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b723700-41ac-4b90-8ed5-b37e196a297b admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:46.980247 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1329/2671] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:59:46.985337 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d31d45-e6bb-4c5c-9a3f-9db4bd3b4881 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:46.996243 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d31d45-e6bb-4c5c-9a3f-9db4bd3b4881 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:47.002496 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8534e-063a-4a59-ac22-e2f5b17d9baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:47.003280 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8534e-063a-4a59-ac22-e2f5b17d9baf admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:47.003570 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8534e-063a-4a59-ac22-e2f5b17d9baf admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:47.003852 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8534e-063a-4a59-ac22-e2f5b17d9baf admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:47.004545 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a8534e-063a-4a59-ac22-e2f5b17d9baf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:47.006281 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a8534e-063a-4a59-ac22-e2f5b17d9baf admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:47.019303 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1343/2672] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:59:47.023611 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b723700-41ac-4b90-8ed5-b37e196a297b admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:47.034470 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b723700-41ac-4b90-8ed5-b37e196a297b admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:47.039123 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a14960-e9bb-43f6-8c30-e2060c72d4ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:47.039788 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a14960-e9bb-43f6-8c30-e2060c72d4ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:47.040073 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a14960-e9bb-43f6-8c30-e2060c72d4ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:47.040355 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a14960-e9bb-43f6-8c30-e2060c72d4ff admin admin] PATH_INFO: `/v3/projects/88de2f155c914cae970d402000a13dbf/users/6752bf18143848d28c9058424161106d/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:47.058548 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a14960-e9bb-43f6-8c30-e2060c72d4ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=88de2f155c914cae970d402000a13dbf, user_id=6752bf18143848d28c9058424161106d, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:47.061178 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a14960-e9bb-43f6-8c30-e2060c72d4ff admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:47.140080 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1330/2673] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:59:47 2022] PUT /identity/v3/projects/88de2f155c914cae970d402000a13dbf/users/6752bf18143848d28c9058424161106d/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:59:47.150160 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eccc0d-a1ff-4125-8eed-b103be03747e None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:47.150446 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eccc0d-a1ff-4125-8eed-b103be03747e None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:47.150725 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eccc0d-a1ff-4125-8eed-b103be03747e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:47.189566 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-a8eccc0d-a1ff-4125-8eed-b103be03747e None None] MFA Rules not processed for user `6752bf18143848d28c9058424161106d`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:59:47.190809 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-a8eccc0d-a1ff-4125-8eed-b103be03747e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:47.265730 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1344/2674] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:59:47 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:59:47.270726 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a14960-e9bb-43f6-8c30-e2060c72d4ff admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:47.299789 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:47.341613 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a14960-e9bb-43f6-8c30-e2060c72d4ff admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:47.347303 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb6cfe-554c-4479-9b2c-ac7f45d00fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:47.348061 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb6cfe-554c-4479-9b2c-ac7f45d00fee admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:47.348346 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb6cfe-554c-4479-9b2c-ac7f45d00fee admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:47.356015 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb6cfe-554c-4479-9b2c-ac7f45d00fee admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:47.356015 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb6cfe-554c-4479-9b2c-ac7f45d00fee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:47.356015 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb6cfe-554c-4479-9b2c-ac7f45d00fee admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:47.446358 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1331/2675] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:59:47 2022] POST /identity/v3/projects => generated 399 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:59:47.450508 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eccc0d-a1ff-4125-8eed-b103be03747e None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:47.462555 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eccc0d-a1ff-4125-8eed-b103be03747e None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:47.481500 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:47.529337 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af92f6-71ad-4907-8b48-50e46b886e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:47.529995 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af92f6-71ad-4907-8b48-50e46b886e3b admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:47.530274 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af92f6-71ad-4907-8b48-50e46b886e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:47.531049 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af92f6-71ad-4907-8b48-50e46b886e3b admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:47.531189 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af92f6-71ad-4907-8b48-50e46b886e3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:47.533909 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af92f6-71ad-4907-8b48-50e46b886e3b admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:47.619449 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1345/2676] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:59:47 2022] POST /identity/v3/users => generated 306 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:59:47.624282 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb6cfe-554c-4479-9b2c-ac7f45d00fee admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:47.635624 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb6cfe-554c-4479-9b2c-ac7f45d00fee admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:47.640511 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3188f-e8fc-4889-bd30-e1bfb7e68779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:47.641758 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3188f-e8fc-4889-bd30-e1bfb7e68779 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:47.642302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3188f-e8fc-4889-bd30-e1bfb7e68779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:47.642588 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3188f-e8fc-4889-bd30-e1bfb7e68779 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:47.643243 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c3188f-e8fc-4889-bd30-e1bfb7e68779 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:47.644881 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c3188f-e8fc-4889-bd30-e1bfb7e68779 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:47.655657 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1332/2677] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:59:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:59:47.660388 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af92f6-71ad-4907-8b48-50e46b886e3b admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:47.671936 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af92f6-71ad-4907-8b48-50e46b886e3b admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:47.677135 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2fec6-6528-4c87-b7f6-4ae34cce08dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:47.677811 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2fec6-6528-4c87-b7f6-4ae34cce08dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:47.678097 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2fec6-6528-4c87-b7f6-4ae34cce08dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:47.678376 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2fec6-6528-4c87-b7f6-4ae34cce08dc admin admin] PATH_INFO: `/v3/projects/d36caa6dafc541869ff86d53b22ebfb8/users/cff19a50095b43128c41f5fda8c776e5/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:47.693837 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce2fec6-6528-4c87-b7f6-4ae34cce08dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d36caa6dafc541869ff86d53b22ebfb8, user_id=cff19a50095b43128c41f5fda8c776e5, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:47.696545 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce2fec6-6528-4c87-b7f6-4ae34cce08dc admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:47.774054 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1346/2678] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:59:47 2022] PUT /identity/v3/projects/d36caa6dafc541869ff86d53b22ebfb8/users/cff19a50095b43128c41f5fda8c776e5/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:59:47.777997 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3188f-e8fc-4889-bd30-e1bfb7e68779 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:47.788698 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3188f-e8fc-4889-bd30-e1bfb7e68779 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:47.844742 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfdf2fa-b846-4468-8f86-4b2cade542a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:47.845299 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdf2fa-b846-4468-8f86-4b2cade542a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:47.845576 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdf2fa-b846-4468-8f86-4b2cade542a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:47.845847 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdf2fa-b846-4468-8f86-4b2cade542a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:47.846485 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfdf2fa-b846-4468-8f86-4b2cade542a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:47.848072 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfdf2fa-b846-4468-8f86-4b2cade542a0 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:47.857844 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1333/2679] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:59:47 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:59:47.865310 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2fec6-6528-4c87-b7f6-4ae34cce08dc admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:47.876250 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2fec6-6528-4c87-b7f6-4ae34cce08dc admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:47.881005 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0accedd9-a69d-4d9e-9a16-1f02ed537eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:47.881666 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0accedd9-a69d-4d9e-9a16-1f02ed537eb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:47.881948 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0accedd9-a69d-4d9e-9a16-1f02ed537eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:47.882222 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0accedd9-a69d-4d9e-9a16-1f02ed537eb7 admin admin] PATH_INFO: `/v3/projects/d36caa6dafc541869ff86d53b22ebfb8/users/cff19a50095b43128c41f5fda8c776e5/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:47.884269 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0accedd9-a69d-4d9e-9a16-1f02ed537eb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d36caa6dafc541869ff86d53b22ebfb8, user_id=cff19a50095b43128c41f5fda8c776e5, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:47.886950 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0accedd9-a69d-4d9e-9a16-1f02ed537eb7 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:47.897974 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1347/2680] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:59:47 2022] PUT /identity/v3/projects/d36caa6dafc541869ff86d53b22ebfb8/users/cff19a50095b43128c41f5fda8c776e5/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:59:47.909747 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c4104-49cc-4651-88b3-700657e55c56 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:47.910044 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c4104-49cc-4651-88b3-700657e55c56 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:47.910362 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c4104-49cc-4651-88b3-700657e55c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:47.942262 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-735c4104-49cc-4651-88b3-700657e55c56 None None] MFA Rules not processed for user `cff19a50095b43128c41f5fda8c776e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:59:47.943720 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-735c4104-49cc-4651-88b3-700657e55c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:48.029655 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1334/2681] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:59:47 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:59:48.040751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0accedd9-a69d-4d9e-9a16-1f02ed537eb7 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:48.042611 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-0accedd9-a69d-4d9e-9a16-1f02ed537eb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:48.103298 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0accedd9-a69d-4d9e-9a16-1f02ed537eb7 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:48.107118 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ef791-99ed-4e98-b763-215d5add79b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:48.107743 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ef791-99ed-4e98-b763-215d5add79b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:48.108028 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ef791-99ed-4e98-b763-215d5add79b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:48.108310 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ef791-99ed-4e98-b763-215d5add79b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:48.118663 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ef791-99ed-4e98-b763-215d5add79b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:48.120100 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ef791-99ed-4e98-b763-215d5add79b0 service cinder] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:48.132639 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1348/2682] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:59:48 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:59:48.347996 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735c4104-49cc-4651-88b3-700657e55c56 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:48.357936 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735c4104-49cc-4651-88b3-700657e55c56 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:48.362276 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc128e-e9f3-48db-b029-a5f7a4f7981f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:48.362907 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc128e-e9f3-48db-b029-a5f7a4f7981f service cinder] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:48.363193 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc128e-e9f3-48db-b029-a5f7a4f7981f service cinder] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:48.363461 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc128e-e9f3-48db-b029-a5f7a4f7981f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:48.380882 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:48.421377 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bc128e-e9f3-48db-b029-a5f7a4f7981f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:48.422852 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bc128e-e9f3-48db-b029-a5f7a4f7981f service cinder] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:48.432694 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1335/2683] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:59:48 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:59:54.995293 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ef791-99ed-4e98-b763-215d5add79b0 service cinder] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:55.059564 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ef791-99ed-4e98-b763-215d5add79b0 service cinder] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:55.064722 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4fa3be-7bcb-4914-86a3-96fbc38ba2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:55.065358 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fa3be-7bcb-4914-86a3-96fbc38ba2d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:55.065642 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fa3be-7bcb-4914-86a3-96fbc38ba2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:55.065933 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fa3be-7bcb-4914-86a3-96fbc38ba2d5 admin admin] PATH_INFO: `/v3/users/6752bf18143848d28c9058424161106d` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:55.067064 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4fa3be-7bcb-4914-86a3-96fbc38ba2d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6752bf18143848d28c9058424161106d)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:55.069040 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4fa3be-7bcb-4914-86a3-96fbc38ba2d5 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:55.088783 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:55.255643 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-3c4fa3be-7bcb-4914-86a3-96fbc38ba2d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6752bf18143848d28c9058424161106d'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:55.361995 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-3c4fa3be-7bcb-4914-86a3-96fbc38ba2d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6752bf18143848d28c9058424161106d'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:55.369710 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-3c4fa3be-7bcb-4914-86a3-96fbc38ba2d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6752bf18143848d28c9058424161106d'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:55.381913 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-3c4fa3be-7bcb-4914-86a3-96fbc38ba2d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6752bf18143848d28c9058424161106d'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:55.385929 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-3c4fa3be-7bcb-4914-86a3-96fbc38ba2d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6752bf18143848d28c9058424161106d'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:55.388613 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1349/2684] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:59:54 2022] DELETE /identity/v3/users/6752bf18143848d28c9058424161106d => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:59:55.393918 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c5eee-0044-44b1-a880-acde5a6e094f None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:55.394518 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c5eee-0044-44b1-a880-acde5a6e094f None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:55.394518 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c5eee-0044-44b1-a880-acde5a6e094f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:55.413923 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-062c5eee-0044-44b1-a880-acde5a6e094f None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:59:55.420372 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-062c5eee-0044-44b1-a880-acde5a6e094f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:55.474686 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1336/2685] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:59:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:59:55.484735 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4fa3be-7bcb-4914-86a3-96fbc38ba2d5 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:55.486403 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-3c4fa3be-7bcb-4914-86a3-96fbc38ba2d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:55.551573 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4fa3be-7bcb-4914-86a3-96fbc38ba2d5 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:55.555302 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c78f8-1aee-4e83-8db6-a346087ce403 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:55.555916 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c78f8-1aee-4e83-8db6-a346087ce403 service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:55.556200 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c78f8-1aee-4e83-8db6-a346087ce403 service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:55.556510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c78f8-1aee-4e83-8db6-a346087ce403 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:55.566190 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c78f8-1aee-4e83-8db6-a346087ce403 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:55.567627 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c78f8-1aee-4e83-8db6-a346087ce403 service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:55.577214 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1350/2686] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:59:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:59:56.060520 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062c5eee-0044-44b1-a880-acde5a6e094f None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:56.061942 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-062c5eee-0044-44b1-a880-acde5a6e094f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:56.077787 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062c5eee-0044-44b1-a880-acde5a6e094f None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:56.080591 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8496705-09d1-4d04-bf7f-f98759508752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:56.081243 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8496705-09d1-4d04-bf7f-f98759508752 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:56.081519 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8496705-09d1-4d04-bf7f-f98759508752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:56.081894 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8496705-09d1-4d04-bf7f-f98759508752 admin admin] PATH_INFO: `/v3/projects/88de2f155c914cae970d402000a13dbf` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:56.083048 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8496705-09d1-4d04-bf7f-f98759508752 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88de2f155c914cae970d402000a13dbf)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:56.084982 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8496705-09d1-4d04-bf7f-f98759508752 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:56.202405 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-c8496705-09d1-4d04-bf7f-f98759508752 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88de2f155c914cae970d402000a13dbf', 'request_id': 'req-c8496705-09d1-4d04-bf7f-f98759508752'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:56.209630 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-c8496705-09d1-4d04-bf7f-f98759508752 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88de2f155c914cae970d402000a13dbf', 'request_id': 'req-c8496705-09d1-4d04-bf7f-f98759508752'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:56.270776 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-c8496705-09d1-4d04-bf7f-f98759508752 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88de2f155c914cae970d402000a13dbf', 'request_id': 'req-c8496705-09d1-4d04-bf7f-f98759508752'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:56.283193 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-c8496705-09d1-4d04-bf7f-f98759508752 admin admin] The token cache is being invalidate because project 88de2f155c914cae970d402000a13dbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:59:56.283486 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-c8496705-09d1-4d04-bf7f-f98759508752 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:56.284213 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-c8496705-09d1-4d04-bf7f-f98759508752 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:56.285892 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1337/2687] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:59:56 2022] DELETE /identity/v3/projects/88de2f155c914cae970d402000a13dbf => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:59:56.289850 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c78f8-1aee-4e83-8db6-a346087ce403 service neutron] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:56.300427 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c78f8-1aee-4e83-8db6-a346087ce403 service neutron] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:56.356506 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda037b2-7b91-4ace-8c3c-d8b4f79176b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:56.357086 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda037b2-7b91-4ace-8c3c-d8b4f79176b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:56.357357 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda037b2-7b91-4ace-8c3c-d8b4f79176b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:56.357637 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda037b2-7b91-4ace-8c3c-d8b4f79176b2 admin admin] PATH_INFO: `/v3/users/cff19a50095b43128c41f5fda8c776e5` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:56.358763 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda037b2-7b91-4ace-8c3c-d8b4f79176b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cff19a50095b43128c41f5fda8c776e5)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:56.361687 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda037b2-7b91-4ace-8c3c-d8b4f79176b2 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:56.583013 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:56.589115 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-cda037b2-7b91-4ace-8c3c-d8b4f79176b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cff19a50095b43128c41f5fda8c776e5'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:56.705226 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-cda037b2-7b91-4ace-8c3c-d8b4f79176b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cff19a50095b43128c41f5fda8c776e5'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:56.712790 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-cda037b2-7b91-4ace-8c3c-d8b4f79176b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cff19a50095b43128c41f5fda8c776e5'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:56.725739 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-cda037b2-7b91-4ace-8c3c-d8b4f79176b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cff19a50095b43128c41f5fda8c776e5'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:56.726492 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-cda037b2-7b91-4ace-8c3c-d8b4f79176b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cff19a50095b43128c41f5fda8c776e5'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:56.735461 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1351/2688] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 05:59:56 2022] DELETE /identity/v3/users/cff19a50095b43128c41f5fda8c776e5 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:59:57.163812 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8496705-09d1-4d04-bf7f-f98759508752 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:57.165346 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-c8496705-09d1-4d04-bf7f-f98759508752 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:57.181684 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8496705-09d1-4d04-bf7f-f98759508752 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:57.227684 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:57.244054 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0e0692-854d-48a9-83b1-bb71d920e2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:57.244719 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0e0692-854d-48a9-83b1-bb71d920e2ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:57.245001 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0e0692-854d-48a9-83b1-bb71d920e2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:57.245276 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0e0692-854d-48a9-83b1-bb71d920e2ce admin admin] PATH_INFO: `/v3/projects/d36caa6dafc541869ff86d53b22ebfb8` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:57.246396 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0e0692-854d-48a9-83b1-bb71d920e2ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=d36caa6dafc541869ff86d53b22ebfb8)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:57.248421 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0e0692-854d-48a9-83b1-bb71d920e2ce admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:57.345601 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-bf0e0692-854d-48a9-83b1-bb71d920e2ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd36caa6dafc541869ff86d53b22ebfb8', 'request_id': 'req-bf0e0692-854d-48a9-83b1-bb71d920e2ce'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:57.351765 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-bf0e0692-854d-48a9-83b1-bb71d920e2ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd36caa6dafc541869ff86d53b22ebfb8', 'request_id': 'req-bf0e0692-854d-48a9-83b1-bb71d920e2ce'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:57.458731 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-bf0e0692-854d-48a9-83b1-bb71d920e2ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd36caa6dafc541869ff86d53b22ebfb8', 'request_id': 'req-bf0e0692-854d-48a9-83b1-bb71d920e2ce'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:57.469545 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-bf0e0692-854d-48a9-83b1-bb71d920e2ce admin admin] The token cache is being invalidate because project d36caa6dafc541869ff86d53b22ebfb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:59:57.471587 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-bf0e0692-854d-48a9-83b1-bb71d920e2ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:57.471587 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-bf0e0692-854d-48a9-83b1-bb71d920e2ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:59:57.472048 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1338/2689] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 05:59:57 2022] DELETE /identity/v3/projects/d36caa6dafc541869ff86d53b22ebfb8 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:59:57.477156 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80aae4-de97-4bac-bac0-b8a4ff3b26f9 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:57.477650 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80aae4-de97-4bac-bac0-b8a4ff3b26f9 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:57.477739 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80aae4-de97-4bac-bac0-b8a4ff3b26f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:57.498564 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-4e80aae4-de97-4bac-bac0-b8a4ff3b26f9 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:59:57.499780 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-4e80aae4-de97-4bac-bac0-b8a4ff3b26f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:57.559431 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1352/2690] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:59:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:59:57.568265 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca4efc-0ad1-4c6b-84b5-31c81320f504 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:57.568654 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca4efc-0ad1-4c6b-84b5-31c81320f504 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:57.568874 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca4efc-0ad1-4c6b-84b5-31c81320f504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:57.588523 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-a2ca4efc-0ad1-4c6b-84b5-31c81320f504 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:59:57.589732 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-a2ca4efc-0ad1-4c6b-84b5-31c81320f504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:57.598082 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1339/2691] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:59:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:59:57.602430 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e80aae4-de97-4bac-bac0-b8a4ff3b26f9 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:57.613025 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e80aae4-de97-4bac-bac0-b8a4ff3b26f9 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:57.617728 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b289547c-4820-4579-ada9-769daabfb542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:57.618333 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b289547c-4820-4579-ada9-769daabfb542 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:57.618612 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b289547c-4820-4579-ada9-769daabfb542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:57.618888 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b289547c-4820-4579-ada9-769daabfb542 admin admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:57.619536 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b289547c-4820-4579-ada9-769daabfb542 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:59:57.619844 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b289547c-4820-4579-ada9-769daabfb542 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:57.621498 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b289547c-4820-4579-ada9-769daabfb542 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:57.634187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1353/2692] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 05:59:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:59:57.638637 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca4efc-0ad1-4c6b-84b5-31c81320f504 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:57.648468 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca4efc-0ad1-4c6b-84b5-31c81320f504 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:57.653214 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef55ba64-5084-47f1-bece-83bcc4afbb28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:57.653810 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55ba64-5084-47f1-bece-83bcc4afbb28 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:57.654093 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55ba64-5084-47f1-bece-83bcc4afbb28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:57.654368 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55ba64-5084-47f1-bece-83bcc4afbb28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:57.654987 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef55ba64-5084-47f1-bece-83bcc4afbb28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:57.658098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef55ba64-5084-47f1-bece-83bcc4afbb28 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:57.752518 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1340/2693] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:59:57 2022] POST /identity/v3/projects => generated 373 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:59:57.756470 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b289547c-4820-4579-ada9-769daabfb542 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:57.766475 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b289547c-4820-4579-ada9-769daabfb542 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:57.822245 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3503314b-3980-4cf9-b248-740236841fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:57.822855 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503314b-3980-4cf9-b248-740236841fb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:57.823137 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503314b-3980-4cf9-b248-740236841fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:57.823414 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503314b-3980-4cf9-b248-740236841fb6 admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:57.824051 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3503314b-3980-4cf9-b248-740236841fb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:57.826004 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3503314b-3980-4cf9-b248-740236841fb6 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:57.934250 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1354/2694] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:59:57 2022] POST /identity/v3/users => generated 287 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:59:57.938572 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef55ba64-5084-47f1-bece-83bcc4afbb28 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:57.949688 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef55ba64-5084-47f1-bece-83bcc4afbb28 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:57.954732 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f5386-4dd0-4159-ae9b-0e7636a0c44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:57.955337 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f5386-4dd0-4159-ae9b-0e7636a0c44c admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:57.955619 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f5386-4dd0-4159-ae9b-0e7636a0c44c admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:57.955893 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f5386-4dd0-4159-ae9b-0e7636a0c44c admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:57.956578 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6f5386-4dd0-4159-ae9b-0e7636a0c44c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:57.958174 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6f5386-4dd0-4159-ae9b-0e7636a0c44c admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:57.969254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1341/2695] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:59:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:59:57.973810 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3503314b-3980-4cf9-b248-740236841fb6 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:57.984109 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3503314b-3980-4cf9-b248-740236841fb6 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:57.989458 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eff625-5e25-42d6-8e63-384342749a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:57.990055 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eff625-5e25-42d6-8e63-384342749a21 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:57.990336 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eff625-5e25-42d6-8e63-384342749a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:57.991091 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eff625-5e25-42d6-8e63-384342749a21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:57.991244 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eff625-5e25-42d6-8e63-384342749a21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:57.992894 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eff625-5e25-42d6-8e63-384342749a21 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:58.005770 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1355/2696] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 05:59:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:59:58.008892 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f5386-4dd0-4159-ae9b-0e7636a0c44c admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:58.019747 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f5386-4dd0-4159-ae9b-0e7636a0c44c admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:58.024513 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a585f375-161b-4206-93c1-ef0accd4fa00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:58.025169 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585f375-161b-4206-93c1-ef0accd4fa00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:58.025453 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585f375-161b-4206-93c1-ef0accd4fa00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:58.025733 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585f375-161b-4206-93c1-ef0accd4fa00 admin admin] PATH_INFO: `/v3/projects/bdba4258659b46bb9b97c5fe664f777c/users/985965744e9a4f27ba440238dabc8f20/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:58.041344 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a585f375-161b-4206-93c1-ef0accd4fa00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdba4258659b46bb9b97c5fe664f777c, user_id=985965744e9a4f27ba440238dabc8f20, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:58.044089 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a585f375-161b-4206-93c1-ef0accd4fa00 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:58.128411 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1342/2697] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 05:59:58 2022] PUT /identity/v3/projects/bdba4258659b46bb9b97c5fe664f777c/users/985965744e9a4f27ba440238dabc8f20/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:59:58.135042 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb6301-9ef9-442a-a539-9ac34c2dd4bb None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:58.135330 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb6301-9ef9-442a-a539-9ac34c2dd4bb None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:58.135614 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb6301-9ef9-442a-a539-9ac34c2dd4bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:58.154847 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-f2fb6301-9ef9-442a-a539-9ac34c2dd4bb None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:59:58.156723 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-f2fb6301-9ef9-442a-a539-9ac34c2dd4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:58.173699 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:58.217816 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1356/2698] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:59:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:59:58.228694 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a585f375-161b-4206-93c1-ef0accd4fa00 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:58.234927 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-a585f375-161b-4206-93c1-ef0accd4fa00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:58.287033 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a585f375-161b-4206-93c1-ef0accd4fa00 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:58.290579 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3436ff0c-d951-4561-8be3-1408721e37f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:58.291178 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436ff0c-d951-4561-8be3-1408721e37f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:58.291457 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436ff0c-d951-4561-8be3-1408721e37f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:58.291749 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436ff0c-d951-4561-8be3-1408721e37f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:58.301353 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3436ff0c-d951-4561-8be3-1408721e37f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:58.302480 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3436ff0c-d951-4561-8be3-1408721e37f9 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:58.312112 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1343/2699] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 05:59:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:59:59.608648 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68943df2-6e39-4c4f-b407-5275a1aa5bf1 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:59.609313 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68943df2-6e39-4c4f-b407-5275a1aa5bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:59.609313 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68943df2-6e39-4c4f-b407-5275a1aa5bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:59.641338 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-68943df2-6e39-4c4f-b407-5275a1aa5bf1 None None] MFA Rules not processed for user `985965744e9a4f27ba440238dabc8f20`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:59:59.642697 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-68943df2-6e39-4c4f-b407-5275a1aa5bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:59.724613 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1357/2700] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 05:59:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:59:59.731275 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3436ff0c-d951-4561-8be3-1408721e37f9 service neutron] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:59.742934 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3436ff0c-d951-4561-8be3-1408721e37f9 service neutron] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:59.748461 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47032ae8-add0-43e2-abcf-83ca7ddc20e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:59.749118 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47032ae8-add0-43e2-abcf-83ca7ddc20e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:59.749404 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47032ae8-add0-43e2-abcf-83ca7ddc20e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:59.749714 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47032ae8-add0-43e2-abcf-83ca7ddc20e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:59.751097 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47032ae8-add0-43e2-abcf-83ca7ddc20e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:59.753127 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47032ae8-add0-43e2-abcf-83ca7ddc20e1 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:59.764720 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:59.884532 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1344/2701] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 05:59:59 2022] POST /identity/v3/projects => generated 371 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:59:59.888759 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68943df2-6e39-4c4f-b407-5275a1aa5bf1 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:59.900999 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68943df2-6e39-4c4f-b407-5275a1aa5bf1 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:59.971583 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5dd09e-41f7-47f1-85e7-8ae183e17e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:59.972336 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5dd09e-41f7-47f1-85e7-8ae183e17e67 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:59.972715 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5dd09e-41f7-47f1-85e7-8ae183e17e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:59.978767 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5dd09e-41f7-47f1-85e7-8ae183e17e67 admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:59.978767 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5dd09e-41f7-47f1-85e7-8ae183e17e67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:59.978767 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5dd09e-41f7-47f1-85e7-8ae183e17e67 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:00.048319 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47032ae8-add0-43e2-abcf-83ca7ddc20e1 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:00.050463 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-47032ae8-add0-43e2-abcf-83ca7ddc20e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:00:00.114378 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1358/2702] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 05:59:59 2022] POST /identity/v3/users => generated 292 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:00:00.115874 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5dd09e-41f7-47f1-85e7-8ae183e17e67 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:00.132309 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5dd09e-41f7-47f1-85e7-8ae183e17e67 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:00.140314 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca4e97b-ae2a-4278-9ee6-2df55ad18398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:00.142568 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4e97b-ae2a-4278-9ee6-2df55ad18398 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:00.142848 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4e97b-ae2a-4278-9ee6-2df55ad18398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:00.143122 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4e97b-ae2a-4278-9ee6-2df55ad18398 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:00.143798 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca4e97b-ae2a-4278-9ee6-2df55ad18398 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:00.145600 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca4e97b-ae2a-4278-9ee6-2df55ad18398 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:00.147372 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47032ae8-add0-43e2-abcf-83ca7ddc20e1 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:00.154593 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16763a76-73a5-4bfe-bcde-3d23a632b851 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:00.155450 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16763a76-73a5-4bfe-bcde-3d23a632b851 service nova] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:00.155824 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16763a76-73a5-4bfe-bcde-3d23a632b851 service nova] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:00.156254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16763a76-73a5-4bfe-bcde-3d23a632b851 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:00.157265 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1359/2703] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 06:00:00 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:00:00.158609 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-16763a76-73a5-4bfe-bcde-3d23a632b851 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:00:00.161448 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca4e97b-ae2a-4278-9ee6-2df55ad18398 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:00.172200 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca4e97b-ae2a-4278-9ee6-2df55ad18398 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:00.174836 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16763a76-73a5-4bfe-bcde-3d23a632b851 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:00.176900 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16763a76-73a5-4bfe-bcde-3d23a632b851 service nova] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:00.181815 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb4788-11f5-4d3a-8e0b-7c27c9d4c871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:00.182516 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb4788-11f5-4d3a-8e0b-7c27c9d4c871 admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:00.182803 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb4788-11f5-4d3a-8e0b-7c27c9d4c871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:00.183078 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb4788-11f5-4d3a-8e0b-7c27c9d4c871 admin admin] PATH_INFO: `/v3/projects/ef9adeb6a7b74d62b37aa5d13ea16ba0/users/90b7462131bf4b05b7664fe37236f185/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:00.191306 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1345/2704] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:00:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:00:00.204033 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb4788-11f5-4d3a-8e0b-7c27c9d4c871 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef9adeb6a7b74d62b37aa5d13ea16ba0, user_id=90b7462131bf4b05b7664fe37236f185, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:00.206932 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb4788-11f5-4d3a-8e0b-7c27c9d4c871 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:00.312011 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1360/2705] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 06:00:00 2022] PUT /identity/v3/projects/ef9adeb6a7b74d62b37aa5d13ea16ba0/users/90b7462131bf4b05b7664fe37236f185/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:00:00.316721 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16763a76-73a5-4bfe-bcde-3d23a632b851 service nova] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:00.328270 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16763a76-73a5-4bfe-bcde-3d23a632b851 service nova] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:00.403361 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc62574-94a7-4319-a858-8594f30742f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:00.404139 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc62574-94a7-4319-a858-8594f30742f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:00.404935 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc62574-94a7-4319-a858-8594f30742f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:00.404935 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc62574-94a7-4319-a858-8594f30742f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:00.405546 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc62574-94a7-4319-a858-8594f30742f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:00.408095 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc62574-94a7-4319-a858-8594f30742f3 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:00.424018 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1346/2706] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 06:00:00 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:00:00.425252 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb4788-11f5-4d3a-8e0b-7c27c9d4c871 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:00.437606 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb4788-11f5-4d3a-8e0b-7c27c9d4c871 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:00.443205 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b29c5db-05f8-4593-8d1e-fd6f16f21713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:00.444052 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29c5db-05f8-4593-8d1e-fd6f16f21713 admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:00.444385 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29c5db-05f8-4593-8d1e-fd6f16f21713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:00.444863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29c5db-05f8-4593-8d1e-fd6f16f21713 admin admin] PATH_INFO: `/v3/projects/ef9adeb6a7b74d62b37aa5d13ea16ba0/users/90b7462131bf4b05b7664fe37236f185/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:00.447863 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b29c5db-05f8-4593-8d1e-fd6f16f21713 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef9adeb6a7b74d62b37aa5d13ea16ba0, user_id=90b7462131bf4b05b7664fe37236f185, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:00.450702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b29c5db-05f8-4593-8d1e-fd6f16f21713 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:00.454321 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:00:00.464803 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1361/2707] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 06:00:00 2022] PUT /identity/v3/projects/ef9adeb6a7b74d62b37aa5d13ea16ba0/users/90b7462131bf4b05b7664fe37236f185/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:00:02.079563 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8ac78-e565-4ae6-9df2-0c4f0773016d None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:02.080147 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8ac78-e565-4ae6-9df2-0c4f0773016d None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:02.080147 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8ac78-e565-4ae6-9df2-0c4f0773016d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:02.109147 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-a6b8ac78-e565-4ae6-9df2-0c4f0773016d None None] MFA Rules not processed for user `90b7462131bf4b05b7664fe37236f185`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:00:02.110390 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-a6b8ac78-e565-4ae6-9df2-0c4f0773016d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:00:02.195531 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:00:02.206630 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1347/2708] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:00:02 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:00:02.218220 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b29c5db-05f8-4593-8d1e-fd6f16f21713 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:02.220925 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-4b29c5db-05f8-4593-8d1e-fd6f16f21713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:00:02.279087 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b29c5db-05f8-4593-8d1e-fd6f16f21713 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:02.282888 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54482c8-7563-4be5-b166-ce9a4c8c7416 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:02.283499 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54482c8-7563-4be5-b166-ce9a4c8c7416 service cinder] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:02.283781 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54482c8-7563-4be5-b166-ce9a4c8c7416 service cinder] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:02.284055 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54482c8-7563-4be5-b166-ce9a4c8c7416 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:02.350464 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54482c8-7563-4be5-b166-ce9a4c8c7416 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:02.351966 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54482c8-7563-4be5-b166-ce9a4c8c7416 service cinder] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:02.365852 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1362/2709] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:00:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:00:09.463189 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b8ac78-e565-4ae6-9df2-0c4f0773016d None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:09.473726 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b8ac78-e565-4ae6-9df2-0c4f0773016d None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:09.477540 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b95ac2b-2b37-40df-9ff8-2da4a99bde57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:09.478246 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b95ac2b-2b37-40df-9ff8-2da4a99bde57 service cinder] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:09.478600 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b95ac2b-2b37-40df-9ff8-2da4a99bde57 service cinder] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:09.478956 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b95ac2b-2b37-40df-9ff8-2da4a99bde57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:09.488817 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b95ac2b-2b37-40df-9ff8-2da4a99bde57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:09.497773 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b95ac2b-2b37-40df-9ff8-2da4a99bde57 service cinder] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:09.500052 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1348/2710] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:00:09 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:00:12.357267 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:INSERT=1,keystone:SELECT=50 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:00:19.495517 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=8 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:00:26.269000 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54482c8-7563-4be5-b166-ce9a4c8c7416 service cinder] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:26.270458 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-f54482c8-7563-4be5-b166-ce9a4c8c7416 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:00:26.329535 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54482c8-7563-4be5-b166-ce9a4c8c7416 service cinder] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:26.333337 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea44ff04-0447-40cb-971b-24bb0e4cd187 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:26.333939 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea44ff04-0447-40cb-971b-24bb0e4cd187 service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:26.334214 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea44ff04-0447-40cb-971b-24bb0e4cd187 service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:26.334486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea44ff04-0447-40cb-971b-24bb0e4cd187 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:26.336167 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-ea44ff04-0447-40cb-971b-24bb0e4cd187 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:00:26.401846 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea44ff04-0447-40cb-971b-24bb0e4cd187 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:26.403281 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea44ff04-0447-40cb-971b-24bb0e4cd187 service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:26.413607 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1363/2711] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:00:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:00:34.459998 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b95ac2b-2b37-40df-9ff8-2da4a99bde57 service cinder] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:34.459998 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-9b95ac2b-2b37-40df-9ff8-2da4a99bde57 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:00:34.470193 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b95ac2b-2b37-40df-9ff8-2da4a99bde57 service cinder] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:34.474749 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df65137-23ae-4eef-bb18-bcb79a78b9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:34.475377 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df65137-23ae-4eef-bb18-bcb79a78b9fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:34.475660 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df65137-23ae-4eef-bb18-bcb79a78b9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:34.475936 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df65137-23ae-4eef-bb18-bcb79a78b9fc admin admin] PATH_INFO: `/v3/users/b92925514b544fbe9e34326f9c48cb24` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:34.490106 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df65137-23ae-4eef-bb18-bcb79a78b9fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b92925514b544fbe9e34326f9c48cb24)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:34.492041 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df65137-23ae-4eef-bb18-bcb79a78b9fc admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:34.724996 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-2df65137-23ae-4eef-bb18-bcb79a78b9fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b92925514b544fbe9e34326f9c48cb24'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:34.732198 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-2df65137-23ae-4eef-bb18-bcb79a78b9fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b92925514b544fbe9e34326f9c48cb24'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:34.732929 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-2df65137-23ae-4eef-bb18-bcb79a78b9fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b92925514b544fbe9e34326f9c48cb24'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:34.835546 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-2df65137-23ae-4eef-bb18-bcb79a78b9fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b92925514b544fbe9e34326f9c48cb24'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:34.848693 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-2df65137-23ae-4eef-bb18-bcb79a78b9fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b92925514b544fbe9e34326f9c48cb24'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:34.854906 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1349/2712] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 06:00:34 2022] DELETE /identity/v3/users/b92925514b544fbe9e34326f9c48cb24 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:00:35.123029 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea44ff04-0447-40cb-971b-24bb0e4cd187 service neutron] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:35.134434 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea44ff04-0447-40cb-971b-24bb0e4cd187 service neutron] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:35.195050 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814aaaf-519b-4593-a23e-ec50c01540b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:35.195694 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814aaaf-519b-4593-a23e-ec50c01540b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:35.196002 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814aaaf-519b-4593-a23e-ec50c01540b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:35.196279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814aaaf-519b-4593-a23e-ec50c01540b0 admin admin] PATH_INFO: `/v3/projects/8e8dbc250d3447ecbd8c7b6df2d10d2c` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:35.207735 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9814aaaf-519b-4593-a23e-ec50c01540b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e8dbc250d3447ecbd8c7b6df2d10d2c)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:35.209804 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9814aaaf-519b-4593-a23e-ec50c01540b0 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:35.353110 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:00:35.356710 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-9814aaaf-519b-4593-a23e-ec50c01540b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e8dbc250d3447ecbd8c7b6df2d10d2c', 'request_id': 'req-9814aaaf-519b-4593-a23e-ec50c01540b0'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:35.363888 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-9814aaaf-519b-4593-a23e-ec50c01540b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e8dbc250d3447ecbd8c7b6df2d10d2c', 'request_id': 'req-9814aaaf-519b-4593-a23e-ec50c01540b0'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:35.573021 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-9814aaaf-519b-4593-a23e-ec50c01540b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e8dbc250d3447ecbd8c7b6df2d10d2c', 'request_id': 'req-9814aaaf-519b-4593-a23e-ec50c01540b0'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:35.584898 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-9814aaaf-519b-4593-a23e-ec50c01540b0 admin admin] The token cache is being invalidate because project 8e8dbc250d3447ecbd8c7b6df2d10d2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:00:35.585391 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-9814aaaf-519b-4593-a23e-ec50c01540b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:35.586380 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-9814aaaf-519b-4593-a23e-ec50c01540b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:35.587915 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1364/2713] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 06:00:35 2022] DELETE /identity/v3/projects/8e8dbc250d3447ecbd8c7b6df2d10d2c => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:00:35.591914 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df65137-23ae-4eef-bb18-bcb79a78b9fc admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:35.593390 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-2df65137-23ae-4eef-bb18-bcb79a78b9fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:00:35.607373 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df65137-23ae-4eef-bb18-bcb79a78b9fc admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:35.666937 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c624bd-4d2b-41ae-8d17-e8b86b480eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:35.667569 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c624bd-4d2b-41ae-8d17-e8b86b480eab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:35.667851 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c624bd-4d2b-41ae-8d17-e8b86b480eab admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:35.668129 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c624bd-4d2b-41ae-8d17-e8b86b480eab admin admin] PATH_INFO: `/v3/users/c3c573cbfb4c4fb0aad5fbbdf7aac245` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:35.681420 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c624bd-4d2b-41ae-8d17-e8b86b480eab admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3c573cbfb4c4fb0aad5fbbdf7aac245)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:35.683357 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c624bd-4d2b-41ae-8d17-e8b86b480eab admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:35.873036 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-43c624bd-4d2b-41ae-8d17-e8b86b480eab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3c573cbfb4c4fb0aad5fbbdf7aac245'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:35.878590 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:00:35.881524 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-43c624bd-4d2b-41ae-8d17-e8b86b480eab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3c573cbfb4c4fb0aad5fbbdf7aac245'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:35.882201 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-43c624bd-4d2b-41ae-8d17-e8b86b480eab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3c573cbfb4c4fb0aad5fbbdf7aac245'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:36.001825 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-43c624bd-4d2b-41ae-8d17-e8b86b480eab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3c573cbfb4c4fb0aad5fbbdf7aac245'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:36.018700 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-43c624bd-4d2b-41ae-8d17-e8b86b480eab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3c573cbfb4c4fb0aad5fbbdf7aac245'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:36.023813 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1350/2714] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 06:00:35 2022] DELETE /identity/v3/users/c3c573cbfb4c4fb0aad5fbbdf7aac245 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:00:36.334594 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814aaaf-519b-4593-a23e-ec50c01540b0 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:36.338388 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-9814aaaf-519b-4593-a23e-ec50c01540b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:00:36.350472 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814aaaf-519b-4593-a23e-ec50c01540b0 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:36.419665 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da0160-8894-4f88-b29e-57e826032c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:36.419665 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da0160-8894-4f88-b29e-57e826032c6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:36.419665 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da0160-8894-4f88-b29e-57e826032c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:36.419665 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da0160-8894-4f88-b29e-57e826032c6e admin admin] PATH_INFO: `/v3/projects/2a6d93b55fbc4d6a83b413cbb085f972` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:36.435490 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72da0160-8894-4f88-b29e-57e826032c6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a6d93b55fbc4d6a83b413cbb085f972)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:36.437509 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72da0160-8894-4f88-b29e-57e826032c6e admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:36.675797 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-72da0160-8894-4f88-b29e-57e826032c6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a6d93b55fbc4d6a83b413cbb085f972', 'request_id': 'req-72da0160-8894-4f88-b29e-57e826032c6e'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:36.681997 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-72da0160-8894-4f88-b29e-57e826032c6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a6d93b55fbc4d6a83b413cbb085f972', 'request_id': 'req-72da0160-8894-4f88-b29e-57e826032c6e'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:36.881991 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-72da0160-8894-4f88-b29e-57e826032c6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a6d93b55fbc4d6a83b413cbb085f972', 'request_id': 'req-72da0160-8894-4f88-b29e-57e826032c6e'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:36.895746 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-72da0160-8894-4f88-b29e-57e826032c6e admin admin] The token cache is being invalidate because project 2a6d93b55fbc4d6a83b413cbb085f972 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:00:36.896054 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-72da0160-8894-4f88-b29e-57e826032c6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:36.896684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-72da0160-8894-4f88-b29e-57e826032c6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:36.898742 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1365/2715] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 06:00:36 2022] DELETE /identity/v3/projects/2a6d93b55fbc4d6a83b413cbb085f972 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:00:40.098999 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c624bd-4d2b-41ae-8d17-e8b86b480eab admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:40.100522 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-43c624bd-4d2b-41ae-8d17-e8b86b480eab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:00:40.114472 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c624bd-4d2b-41ae-8d17-e8b86b480eab admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:40.171522 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a2106-eca7-49ec-8ea5-6e48af732550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:40.172080 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a2106-eca7-49ec-8ea5-6e48af732550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:40.172366 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a2106-eca7-49ec-8ea5-6e48af732550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:40.173470 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a2106-eca7-49ec-8ea5-6e48af732550 admin admin] PATH_INFO: `/v3/users/985965744e9a4f27ba440238dabc8f20` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:40.174802 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2a2106-eca7-49ec-8ea5-6e48af732550 admin admin] RBAC: Authorizing `identity:delete_user(user_id=985965744e9a4f27ba440238dabc8f20)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:40.177192 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2a2106-eca7-49ec-8ea5-6e48af732550 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:40.409078 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-ca2a2106-eca7-49ec-8ea5-6e48af732550 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '985965744e9a4f27ba440238dabc8f20'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:40.415853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-ca2a2106-eca7-49ec-8ea5-6e48af732550 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '985965744e9a4f27ba440238dabc8f20'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:40.416555 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-ca2a2106-eca7-49ec-8ea5-6e48af732550 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '985965744e9a4f27ba440238dabc8f20'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:40.498393 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-ca2a2106-eca7-49ec-8ea5-6e48af732550 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '985965744e9a4f27ba440238dabc8f20'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:40.511273 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-ca2a2106-eca7-49ec-8ea5-6e48af732550 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '985965744e9a4f27ba440238dabc8f20'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:40.517353 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1351/2716] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 06:00:40 2022] DELETE /identity/v3/users/985965744e9a4f27ba440238dabc8f20 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:00:40.762271 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da0160-8894-4f88-b29e-57e826032c6e admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:40.770736 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-72da0160-8894-4f88-b29e-57e826032c6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:00:40.784557 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da0160-8894-4f88-b29e-57e826032c6e admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:40.834586 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:00:40.843301 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f07fd-73b7-43aa-8820-e367677a0e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:40.843966 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f07fd-73b7-43aa-8820-e367677a0e1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:40.850757 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f07fd-73b7-43aa-8820-e367677a0e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:40.850757 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f07fd-73b7-43aa-8820-e367677a0e1a admin admin] PATH_INFO: `/v3/projects/bdba4258659b46bb9b97c5fe664f777c` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:40.850757 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3f07fd-73b7-43aa-8820-e367677a0e1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdba4258659b46bb9b97c5fe664f777c)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:40.850757 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3f07fd-73b7-43aa-8820-e367677a0e1a admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:40.981353 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-1e3f07fd-73b7-43aa-8820-e367677a0e1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdba4258659b46bb9b97c5fe664f777c', 'request_id': 'req-1e3f07fd-73b7-43aa-8820-e367677a0e1a'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:40.988333 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-1e3f07fd-73b7-43aa-8820-e367677a0e1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdba4258659b46bb9b97c5fe664f777c', 'request_id': 'req-1e3f07fd-73b7-43aa-8820-e367677a0e1a'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:41.098889 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-1e3f07fd-73b7-43aa-8820-e367677a0e1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdba4258659b46bb9b97c5fe664f777c', 'request_id': 'req-1e3f07fd-73b7-43aa-8820-e367677a0e1a'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:41.110117 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-1e3f07fd-73b7-43aa-8820-e367677a0e1a admin admin] The token cache is being invalidate because project bdba4258659b46bb9b97c5fe664f777c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:00:41.110556 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-1e3f07fd-73b7-43aa-8820-e367677a0e1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:41.111058 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-1e3f07fd-73b7-43aa-8820-e367677a0e1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:41.112607 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1366/2717] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 06:00:40 2022] DELETE /identity/v3/projects/bdba4258659b46bb9b97c5fe664f777c => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:00:41.116720 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a2106-eca7-49ec-8ea5-6e48af732550 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:41.118218 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-ca2a2106-eca7-49ec-8ea5-6e48af732550 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:00:41.132864 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a2106-eca7-49ec-8ea5-6e48af732550 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:41.185056 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:00:41.200513 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006dca6b-f0b8-4c40-a539-c893bc64322c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:41.205960 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006dca6b-f0b8-4c40-a539-c893bc64322c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:41.205960 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006dca6b-f0b8-4c40-a539-c893bc64322c admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:41.205960 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006dca6b-f0b8-4c40-a539-c893bc64322c admin admin] PATH_INFO: `/v3/users/90b7462131bf4b05b7664fe37236f185` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:41.205960 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006dca6b-f0b8-4c40-a539-c893bc64322c admin admin] RBAC: Authorizing `identity:delete_user(user_id=90b7462131bf4b05b7664fe37236f185)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:41.208224 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006dca6b-f0b8-4c40-a539-c893bc64322c admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:41.473075 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-006dca6b-f0b8-4c40-a539-c893bc64322c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90b7462131bf4b05b7664fe37236f185'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:41.480129 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-006dca6b-f0b8-4c40-a539-c893bc64322c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90b7462131bf4b05b7664fe37236f185'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:41.480813 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-006dca6b-f0b8-4c40-a539-c893bc64322c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90b7462131bf4b05b7664fe37236f185'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:41.578673 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-006dca6b-f0b8-4c40-a539-c893bc64322c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90b7462131bf4b05b7664fe37236f185'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:41.594080 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-006dca6b-f0b8-4c40-a539-c893bc64322c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90b7462131bf4b05b7664fe37236f185'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:41.600966 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1352/2718] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 06:00:41 2022] DELETE /identity/v3/users/90b7462131bf4b05b7664fe37236f185 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:00:41.905398 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f07fd-73b7-43aa-8820-e367677a0e1a admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:41.906889 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-1e3f07fd-73b7-43aa-8820-e367677a0e1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:00:41.920227 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f07fd-73b7-43aa-8820-e367677a0e1a admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:41.979261 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f97628-7943-48e6-a4cf-1d18207d0e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:41.979897 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f97628-7943-48e6-a4cf-1d18207d0e01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:41.980178 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f97628-7943-48e6-a4cf-1d18207d0e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:41.980488 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f97628-7943-48e6-a4cf-1d18207d0e01 admin admin] PATH_INFO: `/v3/projects/ef9adeb6a7b74d62b37aa5d13ea16ba0` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:41.981610 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f97628-7943-48e6-a4cf-1d18207d0e01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef9adeb6a7b74d62b37aa5d13ea16ba0)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:41.983525 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f97628-7943-48e6-a4cf-1d18207d0e01 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:42.093616 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-a2f97628-7943-48e6-a4cf-1d18207d0e01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef9adeb6a7b74d62b37aa5d13ea16ba0', 'request_id': 'req-a2f97628-7943-48e6-a4cf-1d18207d0e01'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:42.099993 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-a2f97628-7943-48e6-a4cf-1d18207d0e01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef9adeb6a7b74d62b37aa5d13ea16ba0', 'request_id': 'req-a2f97628-7943-48e6-a4cf-1d18207d0e01'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:42.190245 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-a2f97628-7943-48e6-a4cf-1d18207d0e01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef9adeb6a7b74d62b37aa5d13ea16ba0', 'request_id': 'req-a2f97628-7943-48e6-a4cf-1d18207d0e01'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:42.201368 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-a2f97628-7943-48e6-a4cf-1d18207d0e01 admin admin] The token cache is being invalidate because project ef9adeb6a7b74d62b37aa5d13ea16ba0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:00:42.201662 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-a2f97628-7943-48e6-a4cf-1d18207d0e01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:42.204567 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-a2f97628-7943-48e6-a4cf-1d18207d0e01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:42.204567 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1367/2719] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 06:00:41 2022] DELETE /identity/v3/projects/ef9adeb6a7b74d62b37aa5d13ea16ba0 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:00:42.209389 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f7d0a-ebfa-49c0-a454-e56f2c608564 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:42.209905 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f7d0a-ebfa-49c0-a454-e56f2c608564 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:42.210019 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f7d0a-ebfa-49c0-a454-e56f2c608564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:42.234325 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-255f7d0a-ebfa-49c0-a454-e56f2c608564 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:00:42.236527 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-255f7d0a-ebfa-49c0-a454-e56f2c608564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:00:42.300515 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1353/2720] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:00:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:00:42.300985 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d90717-0547-4a7d-a722-89b7d1e8290b None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:42.301269 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d90717-0547-4a7d-a722-89b7d1e8290b None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:42.301809 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d90717-0547-4a7d-a722-89b7d1e8290b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:42.311494 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef341b-d40a-4f32-a3a9-90709e769e7b None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:42.311775 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef341b-d40a-4f32-a3a9-90709e769e7b None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:42.312045 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef341b-d40a-4f32-a3a9-90709e769e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:42.322420 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-82d90717-0547-4a7d-a722-89b7d1e8290b None None] MFA Rules not processed for user `89b5395d46bf4d55bbce8e30198eab6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:00:42.323691 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-82d90717-0547-4a7d-a722-89b7d1e8290b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:00:42.331499 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-2eef341b-d40a-4f32-a3a9-90709e769e7b None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:00:42.332699 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-2eef341b-d40a-4f32-a3a9-90709e769e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:00:42.333113 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:00:42.341030 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1354/2721] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:00:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:00:42.351111 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eef341b-d40a-4f32-a3a9-90709e769e7b None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:42.361102 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eef341b-d40a-4f32-a3a9-90709e769e7b None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:42.366047 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08fdfb8-b605-4d22-a636-981432363011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:42.366693 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fdfb8-b605-4d22-a636-981432363011 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:42.366972 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fdfb8-b605-4d22-a636-981432363011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:42.367245 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fdfb8-b605-4d22-a636-981432363011 admin admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:42.367894 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08fdfb8-b605-4d22-a636-981432363011 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:00:42.368195 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08fdfb8-b605-4d22-a636-981432363011 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:42.369910 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08fdfb8-b605-4d22-a636-981432363011 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:42.382621 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1355/2722] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 06:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:00:42.386859 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08fdfb8-b605-4d22-a636-981432363011 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:42.387136 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1368/2723] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:00:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:00:42.399655 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08fdfb8-b605-4d22-a636-981432363011 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:42.404498 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaee6d3-5b3f-44c2-88a6-8fe1879735cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:42.405290 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaee6d3-5b3f-44c2-88a6-8fe1879735cd admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:42.405581 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaee6d3-5b3f-44c2-88a6-8fe1879735cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:42.405942 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaee6d3-5b3f-44c2-88a6-8fe1879735cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:42.406541 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaee6d3-5b3f-44c2-88a6-8fe1879735cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:42.410523 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaee6d3-5b3f-44c2-88a6-8fe1879735cd admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:42.600800 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1356/2724] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 06:00:42 2022] POST /identity/v3/projects => generated 385 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:00:42.601971 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d90717-0547-4a7d-a722-89b7d1e8290b None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:42.612899 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d90717-0547-4a7d-a722-89b7d1e8290b None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:42.642191 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaee6d3-5b3f-44c2-88a6-8fe1879735cd admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:42.645256 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-0aaee6d3-5b3f-44c2-88a6-8fe1879735cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:00:42.676849 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c2b0a-bed8-4f4b-9018-27036e89031f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:42.677465 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c2b0a-bed8-4f4b-9018-27036e89031f admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:42.677702 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:00:42.677830 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c2b0a-bed8-4f4b-9018-27036e89031f admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:42.678031 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c2b0a-bed8-4f4b-9018-27036e89031f admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:42.678628 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c2b0a-bed8-4f4b-9018-27036e89031f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:42.680720 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c2b0a-bed8-4f4b-9018-27036e89031f admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:42.706559 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaee6d3-5b3f-44c2-88a6-8fe1879735cd admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:42.710312 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94465202-d6b1-4eb0-8c3f-992b9d9c7820 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:42.710918 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94465202-d6b1-4eb0-8c3f-992b9d9c7820 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:42.711198 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94465202-d6b1-4eb0-8c3f-992b9d9c7820 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:42.711471 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94465202-d6b1-4eb0-8c3f-992b9d9c7820 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:42.713221 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-94465202-d6b1-4eb0-8c3f-992b9d9c7820 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:00:42.723043 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94465202-d6b1-4eb0-8c3f-992b9d9c7820 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:42.724556 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94465202-d6b1-4eb0-8c3f-992b9d9c7820 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:42.732656 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1357/2725] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:00:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:00:42.773937 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1369/2726] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 06:00:42 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:00:42.778150 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94465202-d6b1-4eb0-8c3f-992b9d9c7820 service neutron] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:42.790437 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94465202-d6b1-4eb0-8c3f-992b9d9c7820 service neutron] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:42.795085 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30abc979-a50c-44b4-8060-18baddb2a803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:42.795688 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30abc979-a50c-44b4-8060-18baddb2a803 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:42.795968 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30abc979-a50c-44b4-8060-18baddb2a803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:42.796242 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30abc979-a50c-44b4-8060-18baddb2a803 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:42.796915 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30abc979-a50c-44b4-8060-18baddb2a803 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:42.798544 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30abc979-a50c-44b4-8060-18baddb2a803 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:42.811351 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1358/2727] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 06:00:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:00:42.812975 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c2b0a-bed8-4f4b-9018-27036e89031f admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:42.823782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c2b0a-bed8-4f4b-9018-27036e89031f admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:42.828545 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82941b5-fca6-41e0-99bb-cb7d311e3783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:42.829937 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82941b5-fca6-41e0-99bb-cb7d311e3783 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:42.830218 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82941b5-fca6-41e0-99bb-cb7d311e3783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:42.830490 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82941b5-fca6-41e0-99bb-cb7d311e3783 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:42.831128 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82941b5-fca6-41e0-99bb-cb7d311e3783 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:42.832783 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82941b5-fca6-41e0-99bb-cb7d311e3783 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:42.843098 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1370/2728] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 06:00:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:00:42.847818 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30abc979-a50c-44b4-8060-18baddb2a803 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:42.857766 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30abc979-a50c-44b4-8060-18baddb2a803 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:42.862618 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1acd0d3-f609-4da2-b654-533f935b9f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:42.863274 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1acd0d3-f609-4da2-b654-533f935b9f78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:42.863556 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1acd0d3-f609-4da2-b654-533f935b9f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:42.863835 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1acd0d3-f609-4da2-b654-533f935b9f78 admin admin] PATH_INFO: `/v3/projects/fc2bafff718147e7b3ba85c49df40e51/users/e67ff78225f54555bc6e57e7934b04d2/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:42.888797 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1acd0d3-f609-4da2-b654-533f935b9f78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc2bafff718147e7b3ba85c49df40e51, user_id=e67ff78225f54555bc6e57e7934b04d2, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:42.892873 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1acd0d3-f609-4da2-b654-533f935b9f78 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:43.006700 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1359/2729] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 06:00:42 2022] PUT /identity/v3/projects/fc2bafff718147e7b3ba85c49df40e51/users/e67ff78225f54555bc6e57e7934b04d2/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:00:43.009396 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44fb7db-32fb-4444-85ca-5da438cba356 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:43.009885 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44fb7db-32fb-4444-85ca-5da438cba356 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:43.010105 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44fb7db-32fb-4444-85ca-5da438cba356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:43.038650 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-d44fb7db-32fb-4444-85ca-5da438cba356 None None] MFA Rules not processed for user `e67ff78225f54555bc6e57e7934b04d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:00:43.040853 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-d44fb7db-32fb-4444-85ca-5da438cba356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:00:43.106660 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1371/2730] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:00:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:00:43.118398 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1acd0d3-f609-4da2-b654-533f935b9f78 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:43.120063 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-e1acd0d3-f609-4da2-b654-533f935b9f78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:00:43.185830 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1acd0d3-f609-4da2-b654-533f935b9f78 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:43.191661 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42afd75b-2ad2-4972-9e97-873e3f7be2ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:43.192366 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42afd75b-2ad2-4972-9e97-873e3f7be2ef service cinder] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:43.192676 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42afd75b-2ad2-4972-9e97-873e3f7be2ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:43.192953 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42afd75b-2ad2-4972-9e97-873e3f7be2ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:43.210526 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42afd75b-2ad2-4972-9e97-873e3f7be2ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:43.210526 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42afd75b-2ad2-4972-9e97-873e3f7be2ef service cinder] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:43.222581 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1360/2731] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:00:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:00:46.538125 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44fb7db-32fb-4444-85ca-5da438cba356 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:46.539575 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-d44fb7db-32fb-4444-85ca-5da438cba356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:00:46.581358 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:00:46.612286 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44fb7db-32fb-4444-85ca-5da438cba356 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:46.616783 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53dcf38-bcec-461f-87c7-7a21f26cce7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:46.617403 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53dcf38-bcec-461f-87c7-7a21f26cce7d service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:46.617686 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53dcf38-bcec-461f-87c7-7a21f26cce7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:46.617987 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53dcf38-bcec-461f-87c7-7a21f26cce7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:46.619782 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-a53dcf38-bcec-461f-87c7-7a21f26cce7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:00:46.630303 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53dcf38-bcec-461f-87c7-7a21f26cce7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:46.631732 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53dcf38-bcec-461f-87c7-7a21f26cce7d service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:46.640516 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1372/2732] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:00:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:00:53.212122 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=55,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:00:55.479280 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42afd75b-2ad2-4972-9e97-873e3f7be2ef service cinder] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:55.544487 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42afd75b-2ad2-4972-9e97-873e3f7be2ef service cinder] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:55.549107 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7642a249-0e47-44c6-9e2d-ba6db763faef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:55.549746 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642a249-0e47-44c6-9e2d-ba6db763faef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:55.550030 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642a249-0e47-44c6-9e2d-ba6db763faef admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:55.550304 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642a249-0e47-44c6-9e2d-ba6db763faef admin admin] PATH_INFO: `/v3/users/e67ff78225f54555bc6e57e7934b04d2` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:55.551519 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7642a249-0e47-44c6-9e2d-ba6db763faef admin admin] RBAC: Authorizing `identity:delete_user(user_id=e67ff78225f54555bc6e57e7934b04d2)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:55.553732 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7642a249-0e47-44c6-9e2d-ba6db763faef admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:55.599716 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53dcf38-bcec-461f-87c7-7a21f26cce7d service neutron] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:55.601195 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-a53dcf38-bcec-461f-87c7-7a21f26cce7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:00:55.673826 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53dcf38-bcec-461f-87c7-7a21f26cce7d service neutron] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:55.680270 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e341f6-63f6-4c95-b68a-1872bb732f10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:55.680900 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e341f6-63f6-4c95-b68a-1872bb732f10 service nova] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:55.681186 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e341f6-63f6-4c95-b68a-1872bb732f10 service nova] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:55.681465 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e341f6-63f6-4c95-b68a-1872bb732f10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:55.683195 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-58e341f6-63f6-4c95-b68a-1872bb732f10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:00:55.695904 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e341f6-63f6-4c95-b68a-1872bb732f10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:55.697523 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e341f6-63f6-4c95-b68a-1872bb732f10 service nova] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:55.708971 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1373/2733] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:00:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:00:55.777373 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-7642a249-0e47-44c6-9e2d-ba6db763faef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e67ff78225f54555bc6e57e7934b04d2'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:55.784411 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-7642a249-0e47-44c6-9e2d-ba6db763faef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e67ff78225f54555bc6e57e7934b04d2'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:55.785152 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-7642a249-0e47-44c6-9e2d-ba6db763faef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e67ff78225f54555bc6e57e7934b04d2'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:55.852428 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-7642a249-0e47-44c6-9e2d-ba6db763faef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e67ff78225f54555bc6e57e7934b04d2'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:55.865093 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-7642a249-0e47-44c6-9e2d-ba6db763faef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e67ff78225f54555bc6e57e7934b04d2'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:55.871251 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1361/2734] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 06:00:55 2022] DELETE /identity/v3/users/e67ff78225f54555bc6e57e7934b04d2 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:00:55.875787 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bfc730-d2c7-471d-85e5-5b1818f5bfb7 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:55.876072 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bfc730-d2c7-471d-85e5-5b1818f5bfb7 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:55.876380 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bfc730-d2c7-471d-85e5-5b1818f5bfb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:55.896525 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-b7bfc730-d2c7-471d-85e5-5b1818f5bfb7 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:00:55.897721 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-b7bfc730-d2c7-471d-85e5-5b1818f5bfb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:00:55.958732 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1374/2735] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:00:55.968747 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7642a249-0e47-44c6-9e2d-ba6db763faef admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:55.970177 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-7642a249-0e47-44c6-9e2d-ba6db763faef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:00:56.039667 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7642a249-0e47-44c6-9e2d-ba6db763faef admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:56.043297 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a6641c-b618-4331-9f33-c910a0b0ed85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:56.043912 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6641c-b618-4331-9f33-c910a0b0ed85 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:56.044340 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6641c-b618-4331-9f33-c910a0b0ed85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:56.044652 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6641c-b618-4331-9f33-c910a0b0ed85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:56.056656 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a6641c-b618-4331-9f33-c910a0b0ed85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:56.058242 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a6641c-b618-4331-9f33-c910a0b0ed85 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:56.076042 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1362/2736] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:00:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:00:56.495295 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bfc730-d2c7-471d-85e5-5b1818f5bfb7 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:56.496941 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-b7bfc730-d2c7-471d-85e5-5b1818f5bfb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:00:56.509964 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bfc730-d2c7-471d-85e5-5b1818f5bfb7 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:56.514705 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d19342-c08f-4daa-b40a-ccdb71bcd5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:56.515462 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d19342-c08f-4daa-b40a-ccdb71bcd5c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:56.515746 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d19342-c08f-4daa-b40a-ccdb71bcd5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:56.516029 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d19342-c08f-4daa-b40a-ccdb71bcd5c0 admin admin] PATH_INFO: `/v3/projects/fc2bafff718147e7b3ba85c49df40e51` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:56.517196 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d19342-c08f-4daa-b40a-ccdb71bcd5c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc2bafff718147e7b3ba85c49df40e51)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:56.521075 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d19342-c08f-4daa-b40a-ccdb71bcd5c0 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:56.638271 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-d4d19342-c08f-4daa-b40a-ccdb71bcd5c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc2bafff718147e7b3ba85c49df40e51', 'request_id': 'req-d4d19342-c08f-4daa-b40a-ccdb71bcd5c0'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:56.639429 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:00:56.645774 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-d4d19342-c08f-4daa-b40a-ccdb71bcd5c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc2bafff718147e7b3ba85c49df40e51', 'request_id': 'req-d4d19342-c08f-4daa-b40a-ccdb71bcd5c0'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:56.739112 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-d4d19342-c08f-4daa-b40a-ccdb71bcd5c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc2bafff718147e7b3ba85c49df40e51', 'request_id': 'req-d4d19342-c08f-4daa-b40a-ccdb71bcd5c0'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:56.750022 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-d4d19342-c08f-4daa-b40a-ccdb71bcd5c0 admin admin] The token cache is being invalidate because project fc2bafff718147e7b3ba85c49df40e51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:00:56.750332 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-d4d19342-c08f-4daa-b40a-ccdb71bcd5c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:56.750966 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-d4d19342-c08f-4daa-b40a-ccdb71bcd5c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:00:56.752505 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1375/2737] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 06:00:56 2022] DELETE /identity/v3/projects/fc2bafff718147e7b3ba85c49df40e51 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:00:56.758124 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab127171-7437-4b2b-8d4f-ca2c7a3fc1e5 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:56.758594 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab127171-7437-4b2b-8d4f-ca2c7a3fc1e5 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:56.758693 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab127171-7437-4b2b-8d4f-ca2c7a3fc1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:56.778019 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-ab127171-7437-4b2b-8d4f-ca2c7a3fc1e5 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:00:56.779209 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-ab127171-7437-4b2b-8d4f-ca2c7a3fc1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:00:56.796988 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:00:56.838980 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1363/2738] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:00:56.846800 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d2f7a-2f1b-44ef-8c4a-8b305236c77b None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:56.847097 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d2f7a-2f1b-44ef-8c4a-8b305236c77b None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:56.847383 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d2f7a-2f1b-44ef-8c4a-8b305236c77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:56.868667 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-882d2f7a-2f1b-44ef-8c4a-8b305236c77b None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:00:56.869885 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-882d2f7a-2f1b-44ef-8c4a-8b305236c77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:00:56.878066 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1376/2739] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:00:56.883286 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab127171-7437-4b2b-8d4f-ca2c7a3fc1e5 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:56.893020 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab127171-7437-4b2b-8d4f-ca2c7a3fc1e5 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:56.898370 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe113db-b03f-48bc-8c53-5377fe8021bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:56.898965 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe113db-b03f-48bc-8c53-5377fe8021bc admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:56.899246 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe113db-b03f-48bc-8c53-5377fe8021bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:56.899550 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe113db-b03f-48bc-8c53-5377fe8021bc admin admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:56.900190 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe113db-b03f-48bc-8c53-5377fe8021bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:00:56.900537 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe113db-b03f-48bc-8c53-5377fe8021bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:56.902131 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe113db-b03f-48bc-8c53-5377fe8021bc admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:56.914323 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1364/2740] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 06:00:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:00:56.918307 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882d2f7a-2f1b-44ef-8c4a-8b305236c77b None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:56.928272 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882d2f7a-2f1b-44ef-8c4a-8b305236c77b None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:56.932994 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a31ce-a06f-4abc-a0bd-80458c9c396b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:56.933590 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a31ce-a06f-4abc-a0bd-80458c9c396b admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:56.933877 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a31ce-a06f-4abc-a0bd-80458c9c396b admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:56.934159 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a31ce-a06f-4abc-a0bd-80458c9c396b admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:56.934785 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a31ce-a06f-4abc-a0bd-80458c9c396b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:56.936729 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a31ce-a06f-4abc-a0bd-80458c9c396b admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:57.083369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1377/2741] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 06:00:56 2022] POST /identity/v3/projects => generated 397 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:00:57.087757 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe113db-b03f-48bc-8c53-5377fe8021bc admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:57.098098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe113db-b03f-48bc-8c53-5377fe8021bc admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:57.155380 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aef865-f40a-4668-9604-c5aa42d229e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:57.155988 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aef865-f40a-4668-9604-c5aa42d229e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:57.156272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aef865-f40a-4668-9604-c5aa42d229e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:57.156582 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aef865-f40a-4668-9604-c5aa42d229e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:57.157216 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aef865-f40a-4668-9604-c5aa42d229e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:57.159147 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aef865-f40a-4668-9604-c5aa42d229e1 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:57.241523 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1365/2742] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 06:00:57 2022] POST /identity/v3/users => generated 299 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:00:57.245487 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a31ce-a06f-4abc-a0bd-80458c9c396b admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:57.256353 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a31ce-a06f-4abc-a0bd-80458c9c396b admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:57.261475 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82359692-da15-4084-b592-682809739fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:57.265029 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82359692-da15-4084-b592-682809739fff admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:57.266070 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82359692-da15-4084-b592-682809739fff admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:57.266369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82359692-da15-4084-b592-682809739fff admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:57.267049 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82359692-da15-4084-b592-682809739fff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:57.269487 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82359692-da15-4084-b592-682809739fff admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:57.283000 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1378/2743] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 06:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:00:57.287221 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aef865-f40a-4668-9604-c5aa42d229e1 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:57.297482 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aef865-f40a-4668-9604-c5aa42d229e1 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:57.303108 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe77827d-e3c7-4c0a-ba5f-0c54cf379aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:57.303705 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77827d-e3c7-4c0a-ba5f-0c54cf379aef admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:57.303988 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77827d-e3c7-4c0a-ba5f-0c54cf379aef admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:57.304264 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77827d-e3c7-4c0a-ba5f-0c54cf379aef admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:57.305041 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe77827d-e3c7-4c0a-ba5f-0c54cf379aef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:57.306631 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe77827d-e3c7-4c0a-ba5f-0c54cf379aef admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:57.318066 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1366/2744] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 06:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:00:57.322268 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82359692-da15-4084-b592-682809739fff admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:57.332094 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82359692-da15-4084-b592-682809739fff admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:57.336899 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc50aac-bc4e-44ba-aac8-80ac361c909d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:57.337481 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc50aac-bc4e-44ba-aac8-80ac361c909d admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:57.337761 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc50aac-bc4e-44ba-aac8-80ac361c909d admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:57.338035 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc50aac-bc4e-44ba-aac8-80ac361c909d admin admin] PATH_INFO: `/v3/projects/bb8d2a16f1714983aab2724c703e461e/users/703d472b64dc4dce9ae0c7503132db52/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:57.352285 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc50aac-bc4e-44ba-aac8-80ac361c909d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb8d2a16f1714983aab2724c703e461e, user_id=703d472b64dc4dce9ae0c7503132db52, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:57.354970 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc50aac-bc4e-44ba-aac8-80ac361c909d admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:57.440952 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1379/2745] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 06:00:57 2022] PUT /identity/v3/projects/bb8d2a16f1714983aab2724c703e461e/users/703d472b64dc4dce9ae0c7503132db52/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:00:57.453178 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60c16d-c54b-476d-aa7a-4f47a8b0b8b9 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:57.453467 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60c16d-c54b-476d-aa7a-4f47a8b0b8b9 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:57.453745 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60c16d-c54b-476d-aa7a-4f47a8b0b8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:57.481137 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-9d60c16d-c54b-476d-aa7a-4f47a8b0b8b9 None None] MFA Rules not processed for user `703d472b64dc4dce9ae0c7503132db52`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:00:57.482341 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-9d60c16d-c54b-476d-aa7a-4f47a8b0b8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:00:57.547873 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1367/2746] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:00:57 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:00:57.560005 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc50aac-bc4e-44ba-aac8-80ac361c909d admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:57.561802 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-edc50aac-bc4e-44ba-aac8-80ac361c909d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:00:57.617969 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc50aac-bc4e-44ba-aac8-80ac361c909d admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:57.621542 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c900479-16e3-4ff6-af60-d417d04e2d14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:57.622518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c900479-16e3-4ff6-af60-d417d04e2d14 service cinder] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:57.622802 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c900479-16e3-4ff6-af60-d417d04e2d14 service cinder] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:57.623080 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c900479-16e3-4ff6-af60-d417d04e2d14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:57.636845 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c900479-16e3-4ff6-af60-d417d04e2d14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:57.638551 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c900479-16e3-4ff6-af60-d417d04e2d14 service cinder] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:57.653504 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1380/2747] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:00:57 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:01:07.544953 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:07.644728 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=59,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:16.578236 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d60c16d-c54b-476d-aa7a-4f47a8b0b8b9 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:16.642268 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d60c16d-c54b-476d-aa7a-4f47a8b0b8b9 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:16.647391 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c4db27-8524-43ef-a248-dce94606a9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:16.648099 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4db27-8524-43ef-a248-dce94606a9a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:16.648383 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4db27-8524-43ef-a248-dce94606a9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:16.648686 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4db27-8524-43ef-a248-dce94606a9a2 admin admin] PATH_INFO: `/v3/users/703d472b64dc4dce9ae0c7503132db52` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:16.649820 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c4db27-8524-43ef-a248-dce94606a9a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=703d472b64dc4dce9ae0c7503132db52)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:16.651731 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c4db27-8524-43ef-a248-dce94606a9a2 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:16.854643 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-16c4db27-8524-43ef-a248-dce94606a9a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '703d472b64dc4dce9ae0c7503132db52'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:01:16.861426 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-16c4db27-8524-43ef-a248-dce94606a9a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '703d472b64dc4dce9ae0c7503132db52'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:01:16.862060 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-16c4db27-8524-43ef-a248-dce94606a9a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '703d472b64dc4dce9ae0c7503132db52'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:01:16.943128 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-16c4db27-8524-43ef-a248-dce94606a9a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '703d472b64dc4dce9ae0c7503132db52'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:01:16.955883 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-16c4db27-8524-43ef-a248-dce94606a9a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '703d472b64dc4dce9ae0c7503132db52'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:01:16.962396 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1368/2748] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 06:01:16 2022] DELETE /identity/v3/users/703d472b64dc4dce9ae0c7503132db52 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:01:16.967049 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b20f27-17a7-4f8c-bdcd-b1240da3b263 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:16.967555 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b20f27-17a7-4f8c-bdcd-b1240da3b263 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:16.967674 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b20f27-17a7-4f8c-bdcd-b1240da3b263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:16.986275 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-81b20f27-17a7-4f8c-bdcd-b1240da3b263 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:01:16.987482 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-81b20f27-17a7-4f8c-bdcd-b1240da3b263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:01:17.046152 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1381/2749] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:01:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:01:17.056003 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c4db27-8524-43ef-a248-dce94606a9a2 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:17.057425 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-16c4db27-8524-43ef-a248-dce94606a9a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:01:17.118069 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c4db27-8524-43ef-a248-dce94606a9a2 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:17.121668 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ccf29-f1fe-47ef-9d1e-ce5956236921 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:17.122268 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ccf29-f1fe-47ef-9d1e-ce5956236921 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:17.122550 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ccf29-f1fe-47ef-9d1e-ce5956236921 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:17.122825 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ccf29-f1fe-47ef-9d1e-ce5956236921 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:17.131608 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9ccf29-f1fe-47ef-9d1e-ce5956236921 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:17.133107 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9ccf29-f1fe-47ef-9d1e-ce5956236921 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:17.142499 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1369/2750] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:01:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:01:17.602379 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b20f27-17a7-4f8c-bdcd-b1240da3b263 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:17.606582 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-81b20f27-17a7-4f8c-bdcd-b1240da3b263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:01:17.617928 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b20f27-17a7-4f8c-bdcd-b1240da3b263 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:17.622683 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7c9454-7dc0-42f9-8c1d-a0afce57e584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:17.623950 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c9454-7dc0-42f9-8c1d-a0afce57e584 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:17.624245 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c9454-7dc0-42f9-8c1d-a0afce57e584 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:17.624735 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c9454-7dc0-42f9-8c1d-a0afce57e584 admin admin] PATH_INFO: `/v3/projects/bb8d2a16f1714983aab2724c703e461e` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:17.625920 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7c9454-7dc0-42f9-8c1d-a0afce57e584 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb8d2a16f1714983aab2724c703e461e)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:17.627963 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7c9454-7dc0-42f9-8c1d-a0afce57e584 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:17.744198 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-8c7c9454-7dc0-42f9-8c1d-a0afce57e584 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb8d2a16f1714983aab2724c703e461e', 'request_id': 'req-8c7c9454-7dc0-42f9-8c1d-a0afce57e584'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:01:17.750332 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-8c7c9454-7dc0-42f9-8c1d-a0afce57e584 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb8d2a16f1714983aab2724c703e461e', 'request_id': 'req-8c7c9454-7dc0-42f9-8c1d-a0afce57e584'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:01:17.834691 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-8c7c9454-7dc0-42f9-8c1d-a0afce57e584 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb8d2a16f1714983aab2724c703e461e', 'request_id': 'req-8c7c9454-7dc0-42f9-8c1d-a0afce57e584'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:01:17.846297 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-8c7c9454-7dc0-42f9-8c1d-a0afce57e584 admin admin] The token cache is being invalidate because project bb8d2a16f1714983aab2724c703e461e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:01:17.846591 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-8c7c9454-7dc0-42f9-8c1d-a0afce57e584 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:01:17.847216 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-8c7c9454-7dc0-42f9-8c1d-a0afce57e584 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:01:17.848735 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1382/2751] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 06:01:17 2022] DELETE /identity/v3/projects/bb8d2a16f1714983aab2724c703e461e => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:01:17.853659 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e08f85-5c5c-4889-86f0-d8598dc04035 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:17.854113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e08f85-5c5c-4889-86f0-d8598dc04035 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:17.854227 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e08f85-5c5c-4889-86f0-d8598dc04035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:17.873350 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-b5e08f85-5c5c-4889-86f0-d8598dc04035 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:01:17.874552 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-b5e08f85-5c5c-4889-86f0-d8598dc04035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:01:17.892001 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:17.933920 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1370/2752] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:01:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:01:17.941662 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fc76d-4cfe-4de1-8d5e-47895e5a16c9 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:17.942069 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fc76d-4cfe-4de1-8d5e-47895e5a16c9 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:17.942358 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fc76d-4cfe-4de1-8d5e-47895e5a16c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:17.963900 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-9c5fc76d-4cfe-4de1-8d5e-47895e5a16c9 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:01:17.965151 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-9c5fc76d-4cfe-4de1-8d5e-47895e5a16c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:01:17.973111 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1383/2753] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:01:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:01:17.977612 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e08f85-5c5c-4889-86f0-d8598dc04035 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:17.987193 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e08f85-5c5c-4889-86f0-d8598dc04035 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:17.991825 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4f25a-5e99-414e-8cbd-cd251fcccf11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:17.992429 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4f25a-5e99-414e-8cbd-cd251fcccf11 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:17.992722 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4f25a-5e99-414e-8cbd-cd251fcccf11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:17.992995 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4f25a-5e99-414e-8cbd-cd251fcccf11 admin admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:17.993633 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f4f25a-5e99-414e-8cbd-cd251fcccf11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:01:17.993993 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f4f25a-5e99-414e-8cbd-cd251fcccf11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:17.995893 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f4f25a-5e99-414e-8cbd-cd251fcccf11 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:18.008199 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1371/2754] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 06:01:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:01:18.012417 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5fc76d-4cfe-4de1-8d5e-47895e5a16c9 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:18.021605 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5fc76d-4cfe-4de1-8d5e-47895e5a16c9 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:18.027530 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afae15e3-92cb-4760-9401-8d3e7e474625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:18.028131 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae15e3-92cb-4760-9401-8d3e7e474625 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:18.028450 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae15e3-92cb-4760-9401-8d3e7e474625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:18.028736 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae15e3-92cb-4760-9401-8d3e7e474625 admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:18.029366 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afae15e3-92cb-4760-9401-8d3e7e474625 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:18.031287 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afae15e3-92cb-4760-9401-8d3e7e474625 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:18.158347 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1384/2755] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 06:01:18 2022] POST /identity/v3/projects => generated 371 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:01:18.163791 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4f25a-5e99-414e-8cbd-cd251fcccf11 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:18.176430 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4f25a-5e99-414e-8cbd-cd251fcccf11 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:18.235557 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc68d479-0f5b-41f6-a56a-e58849b7515e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:18.236163 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68d479-0f5b-41f6-a56a-e58849b7515e admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:18.236480 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68d479-0f5b-41f6-a56a-e58849b7515e admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:18.236761 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68d479-0f5b-41f6-a56a-e58849b7515e admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:18.237396 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc68d479-0f5b-41f6-a56a-e58849b7515e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:18.239318 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc68d479-0f5b-41f6-a56a-e58849b7515e admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:18.352826 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1372/2756] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 06:01:18 2022] POST /identity/v3/users => generated 286 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:01:18.358552 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afae15e3-92cb-4760-9401-8d3e7e474625 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:18.372276 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afae15e3-92cb-4760-9401-8d3e7e474625 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:18.381366 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3513d-b2d6-4f8e-90cc-511e1d1babd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:18.381366 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3513d-b2d6-4f8e-90cc-511e1d1babd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:18.381366 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3513d-b2d6-4f8e-90cc-511e1d1babd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:18.381366 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3513d-b2d6-4f8e-90cc-511e1d1babd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:18.382608 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe3513d-b2d6-4f8e-90cc-511e1d1babd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:18.384011 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe3513d-b2d6-4f8e-90cc-511e1d1babd4 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:18.399159 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1385/2757] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 06:01:18 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:01:18.405163 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc68d479-0f5b-41f6-a56a-e58849b7515e admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:18.416503 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc68d479-0f5b-41f6-a56a-e58849b7515e admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:18.422030 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19168f27-73d2-4623-a6c9-d7ed212a400f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:18.422714 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19168f27-73d2-4623-a6c9-d7ed212a400f admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:18.423018 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19168f27-73d2-4623-a6c9-d7ed212a400f admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:18.423493 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19168f27-73d2-4623-a6c9-d7ed212a400f admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:18.424685 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19168f27-73d2-4623-a6c9-d7ed212a400f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:18.426503 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19168f27-73d2-4623-a6c9-d7ed212a400f admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:18.440809 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1373/2758] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 06:01:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:01:18.445518 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3513d-b2d6-4f8e-90cc-511e1d1babd4 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:18.456366 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3513d-b2d6-4f8e-90cc-511e1d1babd4 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:18.461601 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a6c8f9-e4cc-4c2b-bd32-a617611bd61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:18.463173 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6c8f9-e4cc-4c2b-bd32-a617611bd61c admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:18.463173 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6c8f9-e4cc-4c2b-bd32-a617611bd61c admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:18.463173 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6c8f9-e4cc-4c2b-bd32-a617611bd61c admin admin] PATH_INFO: `/v3/projects/7373b9ca241b4fbba12f2413458072bb/users/3b4870c15b8149908b6c1a99b1f73bc0/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:18.479932 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a6c8f9-e4cc-4c2b-bd32-a617611bd61c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7373b9ca241b4fbba12f2413458072bb, user_id=3b4870c15b8149908b6c1a99b1f73bc0, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:18.482651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a6c8f9-e4cc-4c2b-bd32-a617611bd61c admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:18.571588 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1386/2759] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 06:01:18 2022] PUT /identity/v3/projects/7373b9ca241b4fbba12f2413458072bb/users/3b4870c15b8149908b6c1a99b1f73bc0/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:01:18.577380 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c556a-9951-4802-ab8c-76d1120a04ea None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:18.577912 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c556a-9951-4802-ab8c-76d1120a04ea None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:18.578434 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c556a-9951-4802-ab8c-76d1120a04ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:18.604514 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-dd1c556a-9951-4802-ab8c-76d1120a04ea None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:01:18.606397 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-dd1c556a-9951-4802-ab8c-76d1120a04ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:01:18.677318 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1374/2760] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:01:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:01:18.688639 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a6c8f9-e4cc-4c2b-bd32-a617611bd61c admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:18.690381 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-73a6c8f9-e4cc-4c2b-bd32-a617611bd61c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:01:18.718375 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:18.764119 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a6c8f9-e4cc-4c2b-bd32-a617611bd61c admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:18.767835 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4916d-507b-4642-99f5-cce87ceb1591 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:18.768484 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4916d-507b-4642-99f5-cce87ceb1591 service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:18.768772 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4916d-507b-4642-99f5-cce87ceb1591 service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:18.769882 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4916d-507b-4642-99f5-cce87ceb1591 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:18.781127 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c4916d-507b-4642-99f5-cce87ceb1591 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:18.782563 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c4916d-507b-4642-99f5-cce87ceb1591 service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:18.793483 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1387/2761] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:01:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:01:24.168109 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d43c1-5392-49bf-aa23-8117f305a175 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:24.168713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d43c1-5392-49bf-aa23-8117f305a175 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:24.168713 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d43c1-5392-49bf-aa23-8117f305a175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:24.229147 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-5f8d43c1-5392-49bf-aa23-8117f305a175 None None] MFA Rules not processed for user `3b4870c15b8149908b6c1a99b1f73bc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:01:24.231080 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-5f8d43c1-5392-49bf-aa23-8117f305a175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:01:24.249282 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:24.312646 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1375/2762] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:01:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:01:24.316803 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4916d-507b-4642-99f5-cce87ceb1591 service neutron] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:24.327179 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4916d-507b-4642-99f5-cce87ceb1591 service neutron] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:24.332055 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2475ecb9-7f4a-4dc1-9115-1c5127526bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:24.332734 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475ecb9-7f4a-4dc1-9115-1c5127526bdb admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:24.333018 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475ecb9-7f4a-4dc1-9115-1c5127526bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:24.333299 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475ecb9-7f4a-4dc1-9115-1c5127526bdb admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:24.333936 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2475ecb9-7f4a-4dc1-9115-1c5127526bdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:24.336449 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2475ecb9-7f4a-4dc1-9115-1c5127526bdb admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:24.433060 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1388/2763] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 06:01:24 2022] POST /identity/v3/projects => generated 371 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:01:24.436961 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8d43c1-5392-49bf-aa23-8117f305a175 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:24.448749 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8d43c1-5392-49bf-aa23-8117f305a175 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:24.509287 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf05b1a-1bef-4230-a1fb-4afdbc2376f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:24.509916 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf05b1a-1bef-4230-a1fb-4afdbc2376f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:24.510201 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf05b1a-1bef-4230-a1fb-4afdbc2376f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:24.510478 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf05b1a-1bef-4230-a1fb-4afdbc2376f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:24.511123 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf05b1a-1bef-4230-a1fb-4afdbc2376f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:24.513099 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf05b1a-1bef-4230-a1fb-4afdbc2376f2 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:24.710066 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1376/2764] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 06:01:24 2022] POST /identity/v3/users => generated 292 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:01:24.714130 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2475ecb9-7f4a-4dc1-9115-1c5127526bdb admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:24.725062 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2475ecb9-7f4a-4dc1-9115-1c5127526bdb admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:24.730754 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c942f-10e0-4257-843d-9b44f0548037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:24.731305 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c942f-10e0-4257-843d-9b44f0548037 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:24.731580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c942f-10e0-4257-843d-9b44f0548037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:24.731853 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c942f-10e0-4257-843d-9b44f0548037 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:24.732606 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52c942f-10e0-4257-843d-9b44f0548037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:24.734220 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52c942f-10e0-4257-843d-9b44f0548037 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:24.746686 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1389/2765] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 06:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:01:24.750888 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf05b1a-1bef-4230-a1fb-4afdbc2376f2 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:24.761649 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf05b1a-1bef-4230-a1fb-4afdbc2376f2 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:24.767719 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6490efb-6621-4bec-9644-b48361a7b794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:24.771642 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6490efb-6621-4bec-9644-b48361a7b794 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:24.771933 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6490efb-6621-4bec-9644-b48361a7b794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:24.772373 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6490efb-6621-4bec-9644-b48361a7b794 admin admin] PATH_INFO: `/v3/projects/10c79dadf1674df8b89441d26a09dc87/users/4cc78766dcb34809b4a7fdbead0335de/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:24.790054 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6490efb-6621-4bec-9644-b48361a7b794 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10c79dadf1674df8b89441d26a09dc87, user_id=4cc78766dcb34809b4a7fdbead0335de, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:24.794405 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6490efb-6621-4bec-9644-b48361a7b794 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:24.951305 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1377/2766] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 06:01:24 2022] PUT /identity/v3/projects/10c79dadf1674df8b89441d26a09dc87/users/4cc78766dcb34809b4a7fdbead0335de/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:01:24.955454 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c942f-10e0-4257-843d-9b44f0548037 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:24.966595 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c942f-10e0-4257-843d-9b44f0548037 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:25.038797 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53bf3b7-b00b-4fef-997a-20450154636f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:25.039501 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53bf3b7-b00b-4fef-997a-20450154636f admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:25.039786 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53bf3b7-b00b-4fef-997a-20450154636f admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:25.040065 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53bf3b7-b00b-4fef-997a-20450154636f admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:25.040766 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53bf3b7-b00b-4fef-997a-20450154636f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:25.042376 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53bf3b7-b00b-4fef-997a-20450154636f admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:25.053485 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1390/2767] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 06:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:01:25.057909 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6490efb-6621-4bec-9644-b48361a7b794 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:25.069520 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6490efb-6621-4bec-9644-b48361a7b794 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:25.074576 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab59a80-51d7-4a38-819f-42af4bb29413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:25.075318 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab59a80-51d7-4a38-819f-42af4bb29413 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:25.075662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab59a80-51d7-4a38-819f-42af4bb29413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:25.076004 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab59a80-51d7-4a38-819f-42af4bb29413 admin admin] PATH_INFO: `/v3/projects/10c79dadf1674df8b89441d26a09dc87/users/4cc78766dcb34809b4a7fdbead0335de/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:25.078140 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab59a80-51d7-4a38-819f-42af4bb29413 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10c79dadf1674df8b89441d26a09dc87, user_id=4cc78766dcb34809b4a7fdbead0335de, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:25.080892 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab59a80-51d7-4a38-819f-42af4bb29413 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:25.092629 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1378/2768] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 06:01:25 2022] PUT /identity/v3/projects/10c79dadf1674df8b89441d26a09dc87/users/4cc78766dcb34809b4a7fdbead0335de/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:01:31.031986 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca4e91-9c0f-4f65-9d90-65703fe3d001 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:31.036766 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca4e91-9c0f-4f65-9d90-65703fe3d001 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:31.036766 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca4e91-9c0f-4f65-9d90-65703fe3d001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:31.063679 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-54ca4e91-9c0f-4f65-9d90-65703fe3d001 None None] MFA Rules not processed for user `4cc78766dcb34809b4a7fdbead0335de`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:01:31.064890 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-54ca4e91-9c0f-4f65-9d90-65703fe3d001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:01:31.142302 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1391/2769] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:01:31 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:01:31.156602 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab59a80-51d7-4a38-819f-42af4bb29413 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:31.158491 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-2ab59a80-51d7-4a38-819f-42af4bb29413 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:01:31.242866 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab59a80-51d7-4a38-819f-42af4bb29413 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:31.248855 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36136b9-10f0-452b-bc0e-b22c33f0457f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:31.249478 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36136b9-10f0-452b-bc0e-b22c33f0457f service nova] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:31.249761 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36136b9-10f0-452b-bc0e-b22c33f0457f service nova] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:31.250035 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36136b9-10f0-452b-bc0e-b22c33f0457f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:31.255827 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:31.322869 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36136b9-10f0-452b-bc0e-b22c33f0457f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:31.324533 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36136b9-10f0-452b-bc0e-b22c33f0457f service nova] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:31.335416 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1379/2770] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:01:31 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:01:33.185524 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca4e91-9c0f-4f65-9d90-65703fe3d001 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:33.187079 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-54ca4e91-9c0f-4f65-9d90-65703fe3d001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:01:33.203799 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca4e91-9c0f-4f65-9d90-65703fe3d001 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:33.210711 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0902d15-7dcf-4acb-9f43-97abccddfb35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:33.211425 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0902d15-7dcf-4acb-9f43-97abccddfb35 service nova] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:33.214064 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0902d15-7dcf-4acb-9f43-97abccddfb35 service nova] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:33.214064 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0902d15-7dcf-4acb-9f43-97abccddfb35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:33.214064 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-b0902d15-7dcf-4acb-9f43-97abccddfb35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:01:33.227791 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0902d15-7dcf-4acb-9f43-97abccddfb35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:33.229432 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0902d15-7dcf-4acb-9f43-97abccddfb35 service nova] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:33.234605 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:33.241590 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1392/2771] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:01:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:01:35.546023 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf897f77-917d-4756-8d34-04ece85e782a None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:35.546543 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf897f77-917d-4756-8d34-04ece85e782a None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:35.546649 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf897f77-917d-4756-8d34-04ece85e782a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:35.566933 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-bf897f77-917d-4756-8d34-04ece85e782a None None] MFA Rules not processed for user `3b4870c15b8149908b6c1a99b1f73bc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:01:35.569317 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-bf897f77-917d-4756-8d34-04ece85e782a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:01:35.578817 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1380/2772] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:01:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:01:39.807400 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b318b84f-221e-41c5-a6db-4bdbc6c69547 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:39.807912 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b318b84f-221e-41c5-a6db-4bdbc6c69547 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:39.808034 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b318b84f-221e-41c5-a6db-4bdbc6c69547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:39.828897 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-b318b84f-221e-41c5-a6db-4bdbc6c69547 None None] MFA Rules not processed for user `340c99a3e19343aab956056060a23896`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:01:39.830363 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-b318b84f-221e-41c5-a6db-4bdbc6c69547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:01:39.887129 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1393/2773] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:01:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:01:40.997687 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf897f77-917d-4756-8d34-04ece85e782a None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:40.999208 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-bf897f77-917d-4756-8d34-04ece85e782a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:01:41.082681 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf897f77-917d-4756-8d34-04ece85e782a None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:41.088374 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14271419-e243-48b5-b8af-076490fb0aa8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6778b7c5ad19484b877494f143050405', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:41.089018 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14271419-e243-48b5-b8af-076490fb0aa8 service placement] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:41.089301 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14271419-e243-48b5-b8af-076490fb0aa8 service placement] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:41.089575 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14271419-e243-48b5-b8af-076490fb0aa8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:41.091305 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-14271419-e243-48b5-b8af-076490fb0aa8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:01:41.102798 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14271419-e243-48b5-b8af-076490fb0aa8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:41.104302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14271419-e243-48b5-b8af-076490fb0aa8 service placement] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:41.114272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1381/2774] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:01:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:01:49.880282 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=26 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:51.109895 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=59 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:02:10.167137 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b318b84f-221e-41c5-a6db-4bdbc6c69547 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:02:10.168628 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-b318b84f-221e-41c5-a6db-4bdbc6c69547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:02:10.240692 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b318b84f-221e-41c5-a6db-4bdbc6c69547 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:02:10.245147 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c57de-b223-4e2b-98b8-bfa13f04e412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:02:10.245775 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c57de-b223-4e2b-98b8-bfa13f04e412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:02:10.246056 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c57de-b223-4e2b-98b8-bfa13f04e412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:02:10.246330 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c57de-b223-4e2b-98b8-bfa13f04e412 admin admin] PATH_INFO: `/v3/users/89b5395d46bf4d55bbce8e30198eab6d` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:02:10.247456 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40c57de-b223-4e2b-98b8-bfa13f04e412 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89b5395d46bf4d55bbce8e30198eab6d)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:02:10.249428 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40c57de-b223-4e2b-98b8-bfa13f04e412 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:02:10.501651 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-b40c57de-b223-4e2b-98b8-bfa13f04e412 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89b5395d46bf4d55bbce8e30198eab6d'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:02:10.571271 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-b40c57de-b223-4e2b-98b8-bfa13f04e412 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89b5395d46bf4d55bbce8e30198eab6d'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:02:10.580055 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-b40c57de-b223-4e2b-98b8-bfa13f04e412 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89b5395d46bf4d55bbce8e30198eab6d'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:02:10.592836 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-b40c57de-b223-4e2b-98b8-bfa13f04e412 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89b5395d46bf4d55bbce8e30198eab6d'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:02:10.593809 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-b40c57de-b223-4e2b-98b8-bfa13f04e412 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89b5395d46bf4d55bbce8e30198eab6d'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:02:10.602378 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1394/2775] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 06:02:10 2022] DELETE /identity/v3/users/89b5395d46bf4d55bbce8e30198eab6d => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:02:10.870059 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14271419-e243-48b5-b8af-076490fb0aa8 service placement] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:02:10.883173 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14271419-e243-48b5-b8af-076490fb0aa8 service placement] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:02:10.943873 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afef264f-fb8f-4999-a3b5-82cd8fa469bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:02:10.944540 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef264f-fb8f-4999-a3b5-82cd8fa469bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:02:10.944833 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef264f-fb8f-4999-a3b5-82cd8fa469bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:02:10.945105 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef264f-fb8f-4999-a3b5-82cd8fa469bc admin admin] PATH_INFO: `/v3/projects/4c6ba7a7881546eea41c15fa4c6e0244` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:02:10.946231 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afef264f-fb8f-4999-a3b5-82cd8fa469bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c6ba7a7881546eea41c15fa4c6e0244)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:02:10.948171 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afef264f-fb8f-4999-a3b5-82cd8fa469bc admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:02:11.086171 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-afef264f-fb8f-4999-a3b5-82cd8fa469bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c6ba7a7881546eea41c15fa4c6e0244', 'request_id': 'req-afef264f-fb8f-4999-a3b5-82cd8fa469bc'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:02:11.092238 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-afef264f-fb8f-4999-a3b5-82cd8fa469bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c6ba7a7881546eea41c15fa4c6e0244', 'request_id': 'req-afef264f-fb8f-4999-a3b5-82cd8fa469bc'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:02:11.166047 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-afef264f-fb8f-4999-a3b5-82cd8fa469bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c6ba7a7881546eea41c15fa4c6e0244', 'request_id': 'req-afef264f-fb8f-4999-a3b5-82cd8fa469bc'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:02:11.178488 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-afef264f-fb8f-4999-a3b5-82cd8fa469bc admin admin] The token cache is being invalidate because project 4c6ba7a7881546eea41c15fa4c6e0244 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:02:11.178768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-afef264f-fb8f-4999-a3b5-82cd8fa469bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:02:11.179422 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-afef264f-fb8f-4999-a3b5-82cd8fa469bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:02:11.181076 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1382/2776] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 06:02:10 2022] DELETE /identity/v3/projects/4c6ba7a7881546eea41c15fa4c6e0244 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:02:20.597369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:02:21.174691 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:02:29.505508 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c57de-b223-4e2b-98b8-bfa13f04e412 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:02:29.506944 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-b40c57de-b223-4e2b-98b8-bfa13f04e412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:02:29.563803 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c57de-b223-4e2b-98b8-bfa13f04e412 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:02:29.611007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e85712-b268-4dc2-8ddf-ee1acb85a305 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:02:29.611617 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e85712-b268-4dc2-8ddf-ee1acb85a305 service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:02:29.611899 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e85712-b268-4dc2-8ddf-ee1acb85a305 service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:02:29.612174 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e85712-b268-4dc2-8ddf-ee1acb85a305 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:02:29.613950 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-a4e85712-b268-4dc2-8ddf-ee1acb85a305 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:02:29.677139 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e85712-b268-4dc2-8ddf-ee1acb85a305 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:02:29.679079 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e85712-b268-4dc2-8ddf-ee1acb85a305 service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:02:29.688510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1395/2777] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:02:29 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:02:39.684283 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=70 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:02:50.866587 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afef264f-fb8f-4999-a3b5-82cd8fa469bc admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:02:50.868580 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-afef264f-fb8f-4999-a3b5-82cd8fa469bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:02:51.017381 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afef264f-fb8f-4999-a3b5-82cd8fa469bc admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:02:51.024003 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127769ef-ecdc-44b6-90ed-e880ec94d861 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6778b7c5ad19484b877494f143050405', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:02:51.024722 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127769ef-ecdc-44b6-90ed-e880ec94d861 service placement] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:02:51.025009 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127769ef-ecdc-44b6-90ed-e880ec94d861 service placement] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:02:51.025286 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127769ef-ecdc-44b6-90ed-e880ec94d861 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:02:51.027067 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-127769ef-ecdc-44b6-90ed-e880ec94d861 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:02:51.039714 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127769ef-ecdc-44b6-90ed-e880ec94d861 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:02:51.041321 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127769ef-ecdc-44b6-90ed-e880ec94d861 service placement] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:02:51.051517 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1383/2778] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:02:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:03:01.047102 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=63 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:04:12.544391 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e85712-b268-4dc2-8ddf-ee1acb85a305 service neutron] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:12.545797 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-a4e85712-b268-4dc2-8ddf-ee1acb85a305 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:12.630419 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e85712-b268-4dc2-8ddf-ee1acb85a305 service neutron] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:12.634024 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d46a7bf-18b2-4fef-8af5-fc3752ead383 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:12.634624 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d46a7bf-18b2-4fef-8af5-fc3752ead383 service cinder] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:12.634902 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d46a7bf-18b2-4fef-8af5-fc3752ead383 service cinder] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:12.635178 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d46a7bf-18b2-4fef-8af5-fc3752ead383 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:12.636897 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-6d46a7bf-18b2-4fef-8af5-fc3752ead383 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:12.717667 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d46a7bf-18b2-4fef-8af5-fc3752ead383 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:12.719096 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d46a7bf-18b2-4fef-8af5-fc3752ead383 service cinder] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:12.737695 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1396/2779] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:04:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:04:14.213079 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127769ef-ecdc-44b6-90ed-e880ec94d861 service placement] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:14.214601 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-127769ef-ecdc-44b6-90ed-e880ec94d861 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:14.250353 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127769ef-ecdc-44b6-90ed-e880ec94d861 service placement] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:14.253965 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7a9dba-a299-474b-adf6-8dbdb9d96788 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:14.254566 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7a9dba-a299-474b-adf6-8dbdb9d96788 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:14.254850 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7a9dba-a299-474b-adf6-8dbdb9d96788 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:14.255126 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7a9dba-a299-474b-adf6-8dbdb9d96788 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:14.256871 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-3c7a9dba-a299-474b-adf6-8dbdb9d96788 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:14.339731 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7a9dba-a299-474b-adf6-8dbdb9d96788 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:14.341190 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7a9dba-a299-474b-adf6-8dbdb9d96788 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:14.360363 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1384/2780] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:04:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:04:18.344970 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d46a7bf-18b2-4fef-8af5-fc3752ead383 service cinder] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:18.347879 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-6d46a7bf-18b2-4fef-8af5-fc3752ead383 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:18.364379 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d46a7bf-18b2-4fef-8af5-fc3752ead383 service cinder] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:18.371362 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e88fe90-c7c7-49d9-9010-b6f43116aef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:18.375478 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e88fe90-c7c7-49d9-9010-b6f43116aef7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:18.375478 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e88fe90-c7c7-49d9-9010-b6f43116aef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:18.375478 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e88fe90-c7c7-49d9-9010-b6f43116aef7 admin admin] PATH_INFO: `/v3/users/340c99a3e19343aab956056060a23896` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:18.375478 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e88fe90-c7c7-49d9-9010-b6f43116aef7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=340c99a3e19343aab956056060a23896)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:18.377532 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e88fe90-c7c7-49d9-9010-b6f43116aef7 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:18.635580 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-9e88fe90-c7c7-49d9-9010-b6f43116aef7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '340c99a3e19343aab956056060a23896'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:18.640512 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:04:18.712958 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-9e88fe90-c7c7-49d9-9010-b6f43116aef7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '340c99a3e19343aab956056060a23896'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:18.721503 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-9e88fe90-c7c7-49d9-9010-b6f43116aef7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '340c99a3e19343aab956056060a23896'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:18.735279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-9e88fe90-c7c7-49d9-9010-b6f43116aef7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '340c99a3e19343aab956056060a23896'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:18.737392 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-9e88fe90-c7c7-49d9-9010-b6f43116aef7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '340c99a3e19343aab956056060a23896'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:18.746676 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1397/2781] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 06:04:18 2022] DELETE /identity/v3/users/340c99a3e19343aab956056060a23896 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:04:19.019720 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7a9dba-a299-474b-adf6-8dbdb9d96788 service neutron] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:19.031858 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7a9dba-a299-474b-adf6-8dbdb9d96788 service neutron] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:19.099056 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907526cb-27ab-405b-8333-23fc1ee7f65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:19.099811 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907526cb-27ab-405b-8333-23fc1ee7f65a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:19.100098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907526cb-27ab-405b-8333-23fc1ee7f65a admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:19.100375 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907526cb-27ab-405b-8333-23fc1ee7f65a admin admin] PATH_INFO: `/v3/projects/dd4a461b2a2645908c8229d77e13c743` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:19.101551 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907526cb-27ab-405b-8333-23fc1ee7f65a admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd4a461b2a2645908c8229d77e13c743)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:19.103850 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907526cb-27ab-405b-8333-23fc1ee7f65a admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:19.204092 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:04:19.205397 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-907526cb-27ab-405b-8333-23fc1ee7f65a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd4a461b2a2645908c8229d77e13c743', 'request_id': 'req-907526cb-27ab-405b-8333-23fc1ee7f65a'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:19.211581 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-907526cb-27ab-405b-8333-23fc1ee7f65a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd4a461b2a2645908c8229d77e13c743', 'request_id': 'req-907526cb-27ab-405b-8333-23fc1ee7f65a'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:19.289568 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-907526cb-27ab-405b-8333-23fc1ee7f65a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd4a461b2a2645908c8229d77e13c743', 'request_id': 'req-907526cb-27ab-405b-8333-23fc1ee7f65a'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:19.300630 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-907526cb-27ab-405b-8333-23fc1ee7f65a admin admin] The token cache is being invalidate because project dd4a461b2a2645908c8229d77e13c743 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:04:19.301125 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-907526cb-27ab-405b-8333-23fc1ee7f65a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:19.302961 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-907526cb-27ab-405b-8333-23fc1ee7f65a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:19.304899 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1385/2782] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 06:04:19 2022] DELETE /identity/v3/projects/dd4a461b2a2645908c8229d77e13c743 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:04:19.316786 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5586948c-b0f9-457e-a17c-b45433fe4df7 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:19.317106 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5586948c-b0f9-457e-a17c-b45433fe4df7 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:19.317395 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5586948c-b0f9-457e-a17c-b45433fe4df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:19.374896 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-5586948c-b0f9-457e-a17c-b45433fe4df7 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:04:19.376112 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-5586948c-b0f9-457e-a17c-b45433fe4df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:19.434460 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1398/2783] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:04:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:04:19.443447 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98984438-ea16-4702-aa2e-4df180326f18 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:19.443732 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98984438-ea16-4702-aa2e-4df180326f18 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:19.444006 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98984438-ea16-4702-aa2e-4df180326f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:19.463953 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-98984438-ea16-4702-aa2e-4df180326f18 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:04:19.465224 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-98984438-ea16-4702-aa2e-4df180326f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:19.473170 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1386/2784] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:04:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:04:19.477988 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5586948c-b0f9-457e-a17c-b45433fe4df7 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:19.487614 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5586948c-b0f9-457e-a17c-b45433fe4df7 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:19.492286 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b09436d-6259-49e3-910b-b11fc74bd534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:19.492921 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09436d-6259-49e3-910b-b11fc74bd534 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:19.493204 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09436d-6259-49e3-910b-b11fc74bd534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:19.493508 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09436d-6259-49e3-910b-b11fc74bd534 admin admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:19.494155 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b09436d-6259-49e3-910b-b11fc74bd534 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:04:19.494461 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b09436d-6259-49e3-910b-b11fc74bd534 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:19.496053 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b09436d-6259-49e3-910b-b11fc74bd534 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:19.507632 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1399/2785] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 06:04:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:04:19.511637 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98984438-ea16-4702-aa2e-4df180326f18 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:19.521201 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98984438-ea16-4702-aa2e-4df180326f18 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:19.525765 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f6e4f7-84a3-48d8-850f-43a4efbe75ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:19.526348 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6e4f7-84a3-48d8-850f-43a4efbe75ee admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:19.526626 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6e4f7-84a3-48d8-850f-43a4efbe75ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:19.526893 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6e4f7-84a3-48d8-850f-43a4efbe75ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:19.527528 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f6e4f7-84a3-48d8-850f-43a4efbe75ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:19.529439 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f6e4f7-84a3-48d8-850f-43a4efbe75ee admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:19.644894 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1387/2786] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 06:04:19 2022] POST /identity/v3/projects => generated 403 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:04:19.650137 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b09436d-6259-49e3-910b-b11fc74bd534 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:19.660551 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b09436d-6259-49e3-910b-b11fc74bd534 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:19.717095 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b880b99-0c4a-46c9-ab00-309e502d02ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:19.717625 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b880b99-0c4a-46c9-ab00-309e502d02ef admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:19.717909 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b880b99-0c4a-46c9-ab00-309e502d02ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:19.718187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b880b99-0c4a-46c9-ab00-309e502d02ef admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:19.718809 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b880b99-0c4a-46c9-ab00-309e502d02ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:19.720777 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b880b99-0c4a-46c9-ab00-309e502d02ef admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:19.821302 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1400/2787] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 06:04:19 2022] POST /identity/v3/users => generated 302 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:04:19.826570 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f6e4f7-84a3-48d8-850f-43a4efbe75ee admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:19.838138 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f6e4f7-84a3-48d8-850f-43a4efbe75ee admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:19.842743 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d30efdb-766c-4190-a85f-5aa0d58db361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:19.843372 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d30efdb-766c-4190-a85f-5aa0d58db361 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:19.843652 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d30efdb-766c-4190-a85f-5aa0d58db361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:19.843919 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d30efdb-766c-4190-a85f-5aa0d58db361 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:19.844634 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d30efdb-766c-4190-a85f-5aa0d58db361 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:19.846307 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d30efdb-766c-4190-a85f-5aa0d58db361 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:19.856244 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1388/2788] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 06:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:04:19.860354 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b880b99-0c4a-46c9-ab00-309e502d02ef admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:19.870911 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b880b99-0c4a-46c9-ab00-309e502d02ef admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:19.875610 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e991a6d-dc8b-4320-816e-b4c94a9ffd78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:19.876233 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e991a6d-dc8b-4320-816e-b4c94a9ffd78 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:19.876560 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e991a6d-dc8b-4320-816e-b4c94a9ffd78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:19.876841 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e991a6d-dc8b-4320-816e-b4c94a9ffd78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:19.877485 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e991a6d-dc8b-4320-816e-b4c94a9ffd78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:19.879086 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e991a6d-dc8b-4320-816e-b4c94a9ffd78 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:19.890283 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1401/2789] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 06:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:04:19.894252 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d30efdb-766c-4190-a85f-5aa0d58db361 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:19.904943 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d30efdb-766c-4190-a85f-5aa0d58db361 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:19.909641 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4bb603-13be-4c02-95a1-7eaf55b60d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:19.910274 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4bb603-13be-4c02-95a1-7eaf55b60d2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:19.910551 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4bb603-13be-4c02-95a1-7eaf55b60d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:19.910870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4bb603-13be-4c02-95a1-7eaf55b60d2a admin admin] PATH_INFO: `/v3/projects/3bcfe9735c7e41ed9b4b58133505de0c/users/85152cf6f388440b82bcce1b3d174b79/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:19.924840 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4bb603-13be-4c02-95a1-7eaf55b60d2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bcfe9735c7e41ed9b4b58133505de0c, user_id=85152cf6f388440b82bcce1b3d174b79, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:19.927559 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4bb603-13be-4c02-95a1-7eaf55b60d2a admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:19.997958 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1389/2790] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 06:04:19 2022] PUT /identity/v3/projects/3bcfe9735c7e41ed9b4b58133505de0c/users/85152cf6f388440b82bcce1b3d174b79/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:04:20.008789 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f069ad9e-2cb0-41d0-a417-8e3c8cf8bcab None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:20.009075 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f069ad9e-2cb0-41d0-a417-8e3c8cf8bcab None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:20.009348 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f069ad9e-2cb0-41d0-a417-8e3c8cf8bcab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:20.017429 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:04:20.037867 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-f069ad9e-2cb0-41d0-a417-8e3c8cf8bcab None None] MFA Rules not processed for user `85152cf6f388440b82bcce1b3d174b79`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:04:20.039070 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-f069ad9e-2cb0-41d0-a417-8e3c8cf8bcab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:20.101911 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1402/2791] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:04:20 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:04:20.107404 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4bb603-13be-4c02-95a1-7eaf55b60d2a admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:20.168501 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4bb603-13be-4c02-95a1-7eaf55b60d2a admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:20.173138 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea3bc7-dab8-499c-8465-f535c79b4c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:20.173741 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea3bc7-dab8-499c-8465-f535c79b4c59 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:20.174025 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea3bc7-dab8-499c-8465-f535c79b4c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:20.174300 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea3bc7-dab8-499c-8465-f535c79b4c59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:20.174916 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea3bc7-dab8-499c-8465-f535c79b4c59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:20.177392 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea3bc7-dab8-499c-8465-f535c79b4c59 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:20.285330 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1390/2792] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 06:04:20 2022] POST /identity/v3/projects => generated 405 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:04:20.290254 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f069ad9e-2cb0-41d0-a417-8e3c8cf8bcab None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:20.301040 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f069ad9e-2cb0-41d0-a417-8e3c8cf8bcab None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:20.358059 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d6b63-017e-4ce4-9b63-c16dc31a6656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:20.358669 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d6b63-017e-4ce4-9b63-c16dc31a6656 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:20.358957 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d6b63-017e-4ce4-9b63-c16dc31a6656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:20.359240 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d6b63-017e-4ce4-9b63-c16dc31a6656 admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:20.359931 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d6b63-017e-4ce4-9b63-c16dc31a6656 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:20.361897 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d6b63-017e-4ce4-9b63-c16dc31a6656 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:20.444093 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1403/2793] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 06:04:20 2022] POST /identity/v3/users => generated 309 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:04:20.448112 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea3bc7-dab8-499c-8465-f535c79b4c59 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:20.459038 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea3bc7-dab8-499c-8465-f535c79b4c59 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:20.463618 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ab5008-d489-4f98-9b18-f1fe17f2b0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:20.464212 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab5008-d489-4f98-9b18-f1fe17f2b0b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:20.464534 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab5008-d489-4f98-9b18-f1fe17f2b0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:20.464810 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab5008-d489-4f98-9b18-f1fe17f2b0b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:20.465452 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ab5008-d489-4f98-9b18-f1fe17f2b0b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:20.467185 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ab5008-d489-4f98-9b18-f1fe17f2b0b1 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:20.478442 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1391/2794] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 06:04:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:04:20.482488 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d6b63-017e-4ce4-9b63-c16dc31a6656 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:20.492612 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d6b63-017e-4ce4-9b63-c16dc31a6656 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:20.497282 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3bd89d-863a-4d81-85cd-5277e5dfd0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:20.497930 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3bd89d-863a-4d81-85cd-5277e5dfd0eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:20.498215 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3bd89d-863a-4d81-85cd-5277e5dfd0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:20.498495 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3bd89d-863a-4d81-85cd-5277e5dfd0eb admin admin] PATH_INFO: `/v3/projects/481f15ebbf5741b795be8320560f2791/users/07437205f77544f89d975ec9a49a8e52/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:20.514350 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3bd89d-863a-4d81-85cd-5277e5dfd0eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=481f15ebbf5741b795be8320560f2791, user_id=07437205f77544f89d975ec9a49a8e52, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:20.517140 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3bd89d-863a-4d81-85cd-5277e5dfd0eb admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:20.603999 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1404/2795] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 06:04:20 2022] PUT /identity/v3/projects/481f15ebbf5741b795be8320560f2791/users/07437205f77544f89d975ec9a49a8e52/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:04:20.608066 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ab5008-d489-4f98-9b18-f1fe17f2b0b1 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:20.618168 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ab5008-d489-4f98-9b18-f1fe17f2b0b1 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:20.661527 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:04:20.673765 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecbe96d-b6ee-4261-9c21-792c09a92f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:20.676851 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecbe96d-b6ee-4261-9c21-792c09a92f8f admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:20.676851 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecbe96d-b6ee-4261-9c21-792c09a92f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:20.676851 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecbe96d-b6ee-4261-9c21-792c09a92f8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:20.676851 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecbe96d-b6ee-4261-9c21-792c09a92f8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:20.677379 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecbe96d-b6ee-4261-9c21-792c09a92f8f admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:20.686870 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1392/2796] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 06:04:20 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:04:20.690979 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3bd89d-863a-4d81-85cd-5277e5dfd0eb admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:20.701674 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3bd89d-863a-4d81-85cd-5277e5dfd0eb admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:20.706238 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e58ab-0d05-477d-ad9b-720cb3a0a62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:20.706874 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e58ab-0d05-477d-ad9b-720cb3a0a62e admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:20.707152 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e58ab-0d05-477d-ad9b-720cb3a0a62e admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:20.707427 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e58ab-0d05-477d-ad9b-720cb3a0a62e admin admin] PATH_INFO: `/v3/projects/481f15ebbf5741b795be8320560f2791/users/07437205f77544f89d975ec9a49a8e52/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:20.710147 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56e58ab-0d05-477d-ad9b-720cb3a0a62e admin admin] RBAC: Authorizing `identity:create_grant(project_id=481f15ebbf5741b795be8320560f2791, user_id=07437205f77544f89d975ec9a49a8e52, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:20.712821 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56e58ab-0d05-477d-ad9b-720cb3a0a62e admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:20.722859 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1405/2797] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 06:04:20 2022] PUT /identity/v3/projects/481f15ebbf5741b795be8320560f2791/users/07437205f77544f89d975ec9a49a8e52/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:04:20.734174 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029eb91-f0a7-4533-aac0-6269f2f7d1d7 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:20.734473 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029eb91-f0a7-4533-aac0-6269f2f7d1d7 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:20.734791 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029eb91-f0a7-4533-aac0-6269f2f7d1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:20.763030 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-a029eb91-f0a7-4533-aac0-6269f2f7d1d7 None None] MFA Rules not processed for user `07437205f77544f89d975ec9a49a8e52`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:04:20.764245 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-a029eb91-f0a7-4533-aac0-6269f2f7d1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:20.831263 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1393/2798] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:04:20 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:04:20.842991 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e58ab-0d05-477d-ad9b-720cb3a0a62e admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:20.844726 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-e56e58ab-0d05-477d-ad9b-720cb3a0a62e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:20.899572 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:04:20.904740 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e58ab-0d05-477d-ad9b-720cb3a0a62e admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:20.908497 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a22a3-c0bf-4041-a17c-ace56accdb9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:20.909096 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a22a3-c0bf-4041-a17c-ace56accdb9d service cinder] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:20.913708 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a22a3-c0bf-4041-a17c-ace56accdb9d service cinder] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:20.913708 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a22a3-c0bf-4041-a17c-ace56accdb9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:20.918613 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8a22a3-c0bf-4041-a17c-ace56accdb9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:20.920035 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8a22a3-c0bf-4041-a17c-ace56accdb9d service cinder] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:20.931216 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1406/2799] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:04:20 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:04:21.083631 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a029eb91-f0a7-4533-aac0-6269f2f7d1d7 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:21.148953 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a029eb91-f0a7-4533-aac0-6269f2f7d1d7 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:21.153919 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d96e4f8-473a-42d7-8c20-35ef2ac114ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:21.154606 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d96e4f8-473a-42d7-8c20-35ef2ac114ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:21.154894 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d96e4f8-473a-42d7-8c20-35ef2ac114ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:21.155171 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d96e4f8-473a-42d7-8c20-35ef2ac114ad admin admin] PATH_INFO: `/v3/users/85152cf6f388440b82bcce1b3d174b79` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:21.156310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d96e4f8-473a-42d7-8c20-35ef2ac114ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=85152cf6f388440b82bcce1b3d174b79)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:21.158277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d96e4f8-473a-42d7-8c20-35ef2ac114ad admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:21.371583 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-8d96e4f8-473a-42d7-8c20-35ef2ac114ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85152cf6f388440b82bcce1b3d174b79'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:21.378408 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-8d96e4f8-473a-42d7-8c20-35ef2ac114ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85152cf6f388440b82bcce1b3d174b79'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:21.379068 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-8d96e4f8-473a-42d7-8c20-35ef2ac114ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85152cf6f388440b82bcce1b3d174b79'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:21.453907 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-8d96e4f8-473a-42d7-8c20-35ef2ac114ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85152cf6f388440b82bcce1b3d174b79'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:21.459418 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:04:21.466211 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-8d96e4f8-473a-42d7-8c20-35ef2ac114ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85152cf6f388440b82bcce1b3d174b79'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:21.475399 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1394/2800] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 06:04:21 2022] DELETE /identity/v3/users/85152cf6f388440b82bcce1b3d174b79 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:04:21.477403 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12916cd1-5409-4a75-9b7a-ce5241b970ba None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:21.477923 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12916cd1-5409-4a75-9b7a-ce5241b970ba None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:21.478049 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12916cd1-5409-4a75-9b7a-ce5241b970ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:21.498108 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-12916cd1-5409-4a75-9b7a-ce5241b970ba None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:04:21.499323 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-12916cd1-5409-4a75-9b7a-ce5241b970ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:21.558294 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1407/2801] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:04:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:04:21.568533 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d96e4f8-473a-42d7-8c20-35ef2ac114ad admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:21.570048 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-8d96e4f8-473a-42d7-8c20-35ef2ac114ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:21.629804 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d96e4f8-473a-42d7-8c20-35ef2ac114ad admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:21.633458 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed018d12-5beb-457c-b11c-98ec2fdff352 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:21.634052 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed018d12-5beb-457c-b11c-98ec2fdff352 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:21.634328 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed018d12-5beb-457c-b11c-98ec2fdff352 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:21.634597 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed018d12-5beb-457c-b11c-98ec2fdff352 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:21.643316 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed018d12-5beb-457c-b11c-98ec2fdff352 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:21.644769 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed018d12-5beb-457c-b11c-98ec2fdff352 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:21.654609 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1395/2802] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:04:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:04:22.194735 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12916cd1-5409-4a75-9b7a-ce5241b970ba None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:22.200353 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-12916cd1-5409-4a75-9b7a-ce5241b970ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:22.209102 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12916cd1-5409-4a75-9b7a-ce5241b970ba None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:22.213775 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d7476e-bcf4-44a1-bfc9-14a8452e17b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:22.214418 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7476e-bcf4-44a1-bfc9-14a8452e17b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:22.214700 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7476e-bcf4-44a1-bfc9-14a8452e17b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:22.214976 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7476e-bcf4-44a1-bfc9-14a8452e17b9 admin admin] PATH_INFO: `/v3/projects/3bcfe9735c7e41ed9b4b58133505de0c` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:22.216110 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d7476e-bcf4-44a1-bfc9-14a8452e17b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bcfe9735c7e41ed9b4b58133505de0c)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:22.218068 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d7476e-bcf4-44a1-bfc9-14a8452e17b9 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:22.352256 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-58d7476e-bcf4-44a1-bfc9-14a8452e17b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bcfe9735c7e41ed9b4b58133505de0c', 'request_id': 'req-58d7476e-bcf4-44a1-bfc9-14a8452e17b9'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:22.358342 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-58d7476e-bcf4-44a1-bfc9-14a8452e17b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bcfe9735c7e41ed9b4b58133505de0c', 'request_id': 'req-58d7476e-bcf4-44a1-bfc9-14a8452e17b9'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:22.461162 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-58d7476e-bcf4-44a1-bfc9-14a8452e17b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bcfe9735c7e41ed9b4b58133505de0c', 'request_id': 'req-58d7476e-bcf4-44a1-bfc9-14a8452e17b9'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:22.471607 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-58d7476e-bcf4-44a1-bfc9-14a8452e17b9 admin admin] The token cache is being invalidate because project 3bcfe9735c7e41ed9b4b58133505de0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:04:22.471893 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-58d7476e-bcf4-44a1-bfc9-14a8452e17b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:22.472582 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-58d7476e-bcf4-44a1-bfc9-14a8452e17b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:22.474120 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1408/2803] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 06:04:22 2022] DELETE /identity/v3/projects/3bcfe9735c7e41ed9b4b58133505de0c => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:04:22.478240 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed018d12-5beb-457c-b11c-98ec2fdff352 service neutron] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:22.487997 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed018d12-5beb-457c-b11c-98ec2fdff352 service neutron] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:22.546643 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1042fb9d-065b-4e47-b703-fd20be37c860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:22.547273 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042fb9d-065b-4e47-b703-fd20be37c860 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:22.547555 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042fb9d-065b-4e47-b703-fd20be37c860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:22.547834 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042fb9d-065b-4e47-b703-fd20be37c860 admin admin] PATH_INFO: `/v3/users/07437205f77544f89d975ec9a49a8e52` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:22.548996 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1042fb9d-065b-4e47-b703-fd20be37c860 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07437205f77544f89d975ec9a49a8e52)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:22.550909 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1042fb9d-065b-4e47-b703-fd20be37c860 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:22.777772 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-1042fb9d-065b-4e47-b703-fd20be37c860 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07437205f77544f89d975ec9a49a8e52'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:22.784929 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-1042fb9d-065b-4e47-b703-fd20be37c860 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07437205f77544f89d975ec9a49a8e52'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:22.785725 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-1042fb9d-065b-4e47-b703-fd20be37c860 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07437205f77544f89d975ec9a49a8e52'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:22.878249 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-1042fb9d-065b-4e47-b703-fd20be37c860 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07437205f77544f89d975ec9a49a8e52'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:22.890943 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-1042fb9d-065b-4e47-b703-fd20be37c860 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07437205f77544f89d975ec9a49a8e52'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:22.897545 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1396/2804] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 06:04:22 2022] DELETE /identity/v3/users/07437205f77544f89d975ec9a49a8e52 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:04:23.400558 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d7476e-bcf4-44a1-bfc9-14a8452e17b9 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:23.406382 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-58d7476e-bcf4-44a1-bfc9-14a8452e17b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:23.416327 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d7476e-bcf4-44a1-bfc9-14a8452e17b9 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:23.475001 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3284bc05-6421-4f1c-b88a-8813403ad081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:23.475638 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284bc05-6421-4f1c-b88a-8813403ad081 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:23.475919 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284bc05-6421-4f1c-b88a-8813403ad081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:23.476202 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284bc05-6421-4f1c-b88a-8813403ad081 admin admin] PATH_INFO: `/v3/projects/481f15ebbf5741b795be8320560f2791` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:23.477360 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3284bc05-6421-4f1c-b88a-8813403ad081 admin admin] RBAC: Authorizing `identity:delete_project(project_id=481f15ebbf5741b795be8320560f2791)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:23.479274 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3284bc05-6421-4f1c-b88a-8813403ad081 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:23.495007 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:04:23.587139 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-3284bc05-6421-4f1c-b88a-8813403ad081 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '481f15ebbf5741b795be8320560f2791', 'request_id': 'req-3284bc05-6421-4f1c-b88a-8813403ad081'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:23.593198 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-3284bc05-6421-4f1c-b88a-8813403ad081 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '481f15ebbf5741b795be8320560f2791', 'request_id': 'req-3284bc05-6421-4f1c-b88a-8813403ad081'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:23.706310 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-3284bc05-6421-4f1c-b88a-8813403ad081 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '481f15ebbf5741b795be8320560f2791', 'request_id': 'req-3284bc05-6421-4f1c-b88a-8813403ad081'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:23.717089 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-3284bc05-6421-4f1c-b88a-8813403ad081 admin admin] The token cache is being invalidate because project 481f15ebbf5741b795be8320560f2791 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:04:23.721305 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-3284bc05-6421-4f1c-b88a-8813403ad081 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:23.721305 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-3284bc05-6421-4f1c-b88a-8813403ad081 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:23.721305 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1409/2805] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 06:04:23 2022] DELETE /identity/v3/projects/481f15ebbf5741b795be8320560f2791 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:04:23.724351 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6611bfb-eae0-409c-898b-0710f19f6304 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:23.724901 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6611bfb-eae0-409c-898b-0710f19f6304 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:23.725006 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6611bfb-eae0-409c-898b-0710f19f6304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:23.744186 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-c6611bfb-eae0-409c-898b-0710f19f6304 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:04:23.745432 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-c6611bfb-eae0-409c-898b-0710f19f6304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:23.749618 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:04:23.802716 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1397/2806] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:04:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:04:23.811853 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65f20b-9d5d-4888-84c2-2c71caf797f0 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:23.812144 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65f20b-9d5d-4888-84c2-2c71caf797f0 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:23.812479 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65f20b-9d5d-4888-84c2-2c71caf797f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:23.833037 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-ff65f20b-9d5d-4888-84c2-2c71caf797f0 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:04:23.834279 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-ff65f20b-9d5d-4888-84c2-2c71caf797f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:23.842549 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1410/2807] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:04:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:04:23.852586 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6611bfb-eae0-409c-898b-0710f19f6304 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:23.862244 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6611bfb-eae0-409c-898b-0710f19f6304 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:23.867804 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604238d-73cf-419f-8c65-65a239edeb44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:23.868391 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604238d-73cf-419f-8c65-65a239edeb44 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:23.868696 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604238d-73cf-419f-8c65-65a239edeb44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:23.868964 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604238d-73cf-419f-8c65-65a239edeb44 admin admin] PATH_INFO: `/v3/domains` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:23.869596 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3604238d-73cf-419f-8c65-65a239edeb44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81671) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:04:23.869895 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3604238d-73cf-419f-8c65-65a239edeb44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:23.871500 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3604238d-73cf-419f-8c65-65a239edeb44 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:23.883386 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1398/2808] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 06:04:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:04:23.887969 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff65f20b-9d5d-4888-84c2-2c71caf797f0 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:23.897361 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff65f20b-9d5d-4888-84c2-2c71caf797f0 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:23.902219 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990ef28e-5c5f-4546-a86f-5007d0e90177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:23.902816 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990ef28e-5c5f-4546-a86f-5007d0e90177 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:23.903099 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990ef28e-5c5f-4546-a86f-5007d0e90177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:23.903378 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990ef28e-5c5f-4546-a86f-5007d0e90177 admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:23.904002 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990ef28e-5c5f-4546-a86f-5007d0e90177 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:23.905949 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990ef28e-5c5f-4546-a86f-5007d0e90177 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:24.022724 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1411/2809] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 06:04:23 2022] POST /identity/v3/projects => generated 371 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:04:24.026992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604238d-73cf-419f-8c65-65a239edeb44 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:24.036992 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604238d-73cf-419f-8c65-65a239edeb44 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:24.093412 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7523927-fb77-4c68-b853-b4c8f9574242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:24.094008 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7523927-fb77-4c68-b853-b4c8f9574242 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:24.094288 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7523927-fb77-4c68-b853-b4c8f9574242 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:24.094562 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7523927-fb77-4c68-b853-b4c8f9574242 admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:24.095177 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7523927-fb77-4c68-b853-b4c8f9574242 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:24.097123 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7523927-fb77-4c68-b853-b4c8f9574242 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:24.226741 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1399/2810] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 06:04:24 2022] POST /identity/v3/users => generated 286 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:04:24.231818 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990ef28e-5c5f-4546-a86f-5007d0e90177 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:24.242065 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990ef28e-5c5f-4546-a86f-5007d0e90177 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:24.247603 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3189be-c448-45cb-bbe3-78c80a0a8675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:24.248208 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3189be-c448-45cb-bbe3-78c80a0a8675 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:24.248806 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3189be-c448-45cb-bbe3-78c80a0a8675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:24.249107 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3189be-c448-45cb-bbe3-78c80a0a8675 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:24.249740 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3189be-c448-45cb-bbe3-78c80a0a8675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:24.251344 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3189be-c448-45cb-bbe3-78c80a0a8675 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:24.261479 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1412/2811] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 06:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:04:24.265302 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7523927-fb77-4c68-b853-b4c8f9574242 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:24.275328 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7523927-fb77-4c68-b853-b4c8f9574242 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:24.279969 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d462c868-3083-44a6-a035-bba71b59f5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:24.280603 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462c868-3083-44a6-a035-bba71b59f5eb admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:24.280885 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462c868-3083-44a6-a035-bba71b59f5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:24.281158 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462c868-3083-44a6-a035-bba71b59f5eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:24.281791 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d462c868-3083-44a6-a035-bba71b59f5eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:24.283373 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d462c868-3083-44a6-a035-bba71b59f5eb admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:24.294483 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1400/2812] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 06:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:04:24.298351 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3189be-c448-45cb-bbe3-78c80a0a8675 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:24.309471 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3189be-c448-45cb-bbe3-78c80a0a8675 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:24.314001 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd3c10-5972-41e7-a95f-4874a4d56601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:24.314645 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd3c10-5972-41e7-a95f-4874a4d56601 admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:24.314933 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd3c10-5972-41e7-a95f-4874a4d56601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:24.315211 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd3c10-5972-41e7-a95f-4874a4d56601 admin admin] PATH_INFO: `/v3/projects/22dfb3c6720b4eae93360a4a544f5eac/users/31f0e7704364458f88bc044ae91af259/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:24.330031 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cd3c10-5972-41e7-a95f-4874a4d56601 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22dfb3c6720b4eae93360a4a544f5eac, user_id=31f0e7704364458f88bc044ae91af259, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:24.332706 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cd3c10-5972-41e7-a95f-4874a4d56601 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:24.415778 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1413/2813] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 06:04:24 2022] PUT /identity/v3/projects/22dfb3c6720b4eae93360a4a544f5eac/users/31f0e7704364458f88bc044ae91af259/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:04:24.425919 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eddc2ab-83dc-40e7-9492-d0c37b0aaab6 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:24.426207 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eddc2ab-83dc-40e7-9492-d0c37b0aaab6 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:24.426487 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eddc2ab-83dc-40e7-9492-d0c37b0aaab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:24.454764 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-8eddc2ab-83dc-40e7-9492-d0c37b0aaab6 None None] MFA Rules not processed for user `31f0e7704364458f88bc044ae91af259`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:04:24.456031 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-8eddc2ab-83dc-40e7-9492-d0c37b0aaab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:24.518537 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1401/2814] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:04:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:04:24.523124 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd3c10-5972-41e7-a95f-4874a4d56601 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:24.584379 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd3c10-5972-41e7-a95f-4874a4d56601 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:24.589527 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0852fa-8091-401d-861c-3a8eb14f89fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:24.590126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0852fa-8091-401d-861c-3a8eb14f89fa admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:24.590409 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0852fa-8091-401d-861c-3a8eb14f89fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:24.590688 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0852fa-8091-401d-861c-3a8eb14f89fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:24.591315 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0852fa-8091-401d-861c-3a8eb14f89fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:24.593253 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0852fa-8091-401d-861c-3a8eb14f89fa admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:24.707565 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1414/2815] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 06:04:24 2022] POST /identity/v3/projects => generated 371 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:04:24.712294 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eddc2ab-83dc-40e7-9492-d0c37b0aaab6 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:24.718936 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:04:24.725030 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eddc2ab-83dc-40e7-9492-d0c37b0aaab6 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:24.779376 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf800d5-583a-4a30-b04b-1d47d963aafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:24.780016 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf800d5-583a-4a30-b04b-1d47d963aafc admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:24.780299 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf800d5-583a-4a30-b04b-1d47d963aafc admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:24.780607 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf800d5-583a-4a30-b04b-1d47d963aafc admin admin] PATH_INFO: `/v3/users` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:24.781229 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf800d5-583a-4a30-b04b-1d47d963aafc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:24.783141 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf800d5-583a-4a30-b04b-1d47d963aafc admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:24.872097 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1402/2816] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 06:04:24 2022] POST /identity/v3/users => generated 292 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:04:24.876003 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0852fa-8091-401d-861c-3a8eb14f89fa admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:24.887074 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0852fa-8091-401d-861c-3a8eb14f89fa admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:24.891955 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67213b2-c468-4a2d-a610-7d117c2cc161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:24.892596 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67213b2-c468-4a2d-a610-7d117c2cc161 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:24.892886 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67213b2-c468-4a2d-a610-7d117c2cc161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:24.893166 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67213b2-c468-4a2d-a610-7d117c2cc161 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:24.893856 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67213b2-c468-4a2d-a610-7d117c2cc161 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:24.895456 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67213b2-c468-4a2d-a610-7d117c2cc161 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:24.906707 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1415/2817] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 06:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:04:24.910778 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf800d5-583a-4a30-b04b-1d47d963aafc admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:24.921191 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf800d5-583a-4a30-b04b-1d47d963aafc admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:24.925906 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac91b7-ea3e-4588-b4c8-185ebba1149a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:24.926555 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac91b7-ea3e-4588-b4c8-185ebba1149a admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:24.926840 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac91b7-ea3e-4588-b4c8-185ebba1149a admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:24.927120 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac91b7-ea3e-4588-b4c8-185ebba1149a admin admin] PATH_INFO: `/v3/projects/b5bf0e35ed414f8e9c36191994188fcb/users/d089e03539da47a29c2bba195d2c8105/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:24.942327 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ac91b7-ea3e-4588-b4c8-185ebba1149a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5bf0e35ed414f8e9c36191994188fcb, user_id=d089e03539da47a29c2bba195d2c8105, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:24.945040 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ac91b7-ea3e-4588-b4c8-185ebba1149a admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:25.056328 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1403/2818] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 06:04:24 2022] PUT /identity/v3/projects/b5bf0e35ed414f8e9c36191994188fcb/users/d089e03539da47a29c2bba195d2c8105/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:04:25.060589 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67213b2-c468-4a2d-a610-7d117c2cc161 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:25.071084 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67213b2-c468-4a2d-a610-7d117c2cc161 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:25.084955 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:04:25.135014 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959bd56-cd08-4144-89cb-b570a8c60401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:25.135623 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959bd56-cd08-4144-89cb-b570a8c60401 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:25.135908 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959bd56-cd08-4144-89cb-b570a8c60401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:25.136182 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959bd56-cd08-4144-89cb-b570a8c60401 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:25.136873 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8959bd56-cd08-4144-89cb-b570a8c60401 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:25.140770 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8959bd56-cd08-4144-89cb-b570a8c60401 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:25.148120 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1416/2819] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 06:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:04:25.153324 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac91b7-ea3e-4588-b4c8-185ebba1149a admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:25.163666 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac91b7-ea3e-4588-b4c8-185ebba1149a admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:25.168509 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac8397-2c2f-4851-ad1b-ff324186d90f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:25.169573 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac8397-2c2f-4851-ad1b-ff324186d90f admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:25.169573 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac8397-2c2f-4851-ad1b-ff324186d90f admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:25.169827 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac8397-2c2f-4851-ad1b-ff324186d90f admin admin] PATH_INFO: `/v3/projects/b5bf0e35ed414f8e9c36191994188fcb/users/d089e03539da47a29c2bba195d2c8105/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:25.171763 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac8397-2c2f-4851-ad1b-ff324186d90f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5bf0e35ed414f8e9c36191994188fcb, user_id=d089e03539da47a29c2bba195d2c8105, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:25.174546 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac8397-2c2f-4851-ad1b-ff324186d90f admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:25.186644 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1404/2820] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 06:04:25 2022] PUT /identity/v3/projects/b5bf0e35ed414f8e9c36191994188fcb/users/d089e03539da47a29c2bba195d2c8105/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:04:25.250074 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e1b03-129d-4b93-bccf-9e9fd832e6d2 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:25.250361 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e1b03-129d-4b93-bccf-9e9fd832e6d2 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:25.255331 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e1b03-129d-4b93-bccf-9e9fd832e6d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:25.278797 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-626e1b03-129d-4b93-bccf-9e9fd832e6d2 None None] MFA Rules not processed for user `d089e03539da47a29c2bba195d2c8105`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:04:25.279994 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-626e1b03-129d-4b93-bccf-9e9fd832e6d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:25.346441 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1417/2821] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:04:25 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:04:25.357534 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac8397-2c2f-4851-ad1b-ff324186d90f admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:25.359303 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-b2ac8397-2c2f-4851-ad1b-ff324186d90f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:25.416697 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac8397-2c2f-4851-ad1b-ff324186d90f admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:25.420197 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1834b167-371d-4d04-92b0-8b0e7e655df4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:25.420821 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1834b167-371d-4d04-92b0-8b0e7e655df4 service cinder] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:25.421101 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1834b167-371d-4d04-92b0-8b0e7e655df4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:25.421373 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1834b167-371d-4d04-92b0-8b0e7e655df4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:25.430689 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1834b167-371d-4d04-92b0-8b0e7e655df4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:25.433096 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1834b167-371d-4d04-92b0-8b0e7e655df4 service cinder] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:25.442387 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1405/2822] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:04:25 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:04:26.427034 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626e1b03-129d-4b93-bccf-9e9fd832e6d2 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:26.488559 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626e1b03-129d-4b93-bccf-9e9fd832e6d2 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:26.493067 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e04f6-76e7-46fa-b1f5-0b09f245d765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:26.493686 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e04f6-76e7-46fa-b1f5-0b09f245d765 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:26.493967 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e04f6-76e7-46fa-b1f5-0b09f245d765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:26.494239 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e04f6-76e7-46fa-b1f5-0b09f245d765 admin admin] PATH_INFO: `/v3/users/31f0e7704364458f88bc044ae91af259` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:26.495336 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e04f6-76e7-46fa-b1f5-0b09f245d765 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31f0e7704364458f88bc044ae91af259)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:26.497270 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e04f6-76e7-46fa-b1f5-0b09f245d765 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:26.746571 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:04:26.755209 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-7c8e04f6-76e7-46fa-b1f5-0b09f245d765 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31f0e7704364458f88bc044ae91af259'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:26.849251 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-7c8e04f6-76e7-46fa-b1f5-0b09f245d765 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31f0e7704364458f88bc044ae91af259'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:26.856898 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-7c8e04f6-76e7-46fa-b1f5-0b09f245d765 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31f0e7704364458f88bc044ae91af259'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:26.869726 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-7c8e04f6-76e7-46fa-b1f5-0b09f245d765 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31f0e7704364458f88bc044ae91af259'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:26.870397 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-7c8e04f6-76e7-46fa-b1f5-0b09f245d765 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31f0e7704364458f88bc044ae91af259'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:26.877570 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1418/2823] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 06:04:26 2022] DELETE /identity/v3/users/31f0e7704364458f88bc044ae91af259 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:04:26.882684 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b1d92-dead-4c56-a330-f37211727430 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:26.883170 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b1d92-dead-4c56-a330-f37211727430 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:26.883291 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b1d92-dead-4c56-a330-f37211727430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:26.903088 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-326b1d92-dead-4c56-a330-f37211727430 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:04:26.904596 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-326b1d92-dead-4c56-a330-f37211727430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:26.955680 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:04:26.963899 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1406/2824] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:04:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:04:26.973679 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e04f6-76e7-46fa-b1f5-0b09f245d765 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:26.975164 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-7c8e04f6-76e7-46fa-b1f5-0b09f245d765 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:27.033385 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e04f6-76e7-46fa-b1f5-0b09f245d765 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:27.036992 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17da2b2e-893a-4b1b-be1f-e9fe62b9bf01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:27.037594 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da2b2e-893a-4b1b-be1f-e9fe62b9bf01 service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:27.037877 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da2b2e-893a-4b1b-be1f-e9fe62b9bf01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:27.038152 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da2b2e-893a-4b1b-be1f-e9fe62b9bf01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:27.047714 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17da2b2e-893a-4b1b-be1f-e9fe62b9bf01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:27.049176 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17da2b2e-893a-4b1b-be1f-e9fe62b9bf01 service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:27.058835 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1419/2825] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:04:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:04:27.480247 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b1d92-dead-4c56-a330-f37211727430 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:27.481518 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-326b1d92-dead-4c56-a330-f37211727430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:27.495270 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b1d92-dead-4c56-a330-f37211727430 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:27.501031 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946ac741-a674-49bc-95ac-dc933f64eba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:27.501680 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ac741-a674-49bc-95ac-dc933f64eba2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:27.501963 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ac741-a674-49bc-95ac-dc933f64eba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:27.502244 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ac741-a674-49bc-95ac-dc933f64eba2 admin admin] PATH_INFO: `/v3/projects/22dfb3c6720b4eae93360a4a544f5eac` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:27.503440 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946ac741-a674-49bc-95ac-dc933f64eba2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22dfb3c6720b4eae93360a4a544f5eac)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:27.505489 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946ac741-a674-49bc-95ac-dc933f64eba2 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:27.639386 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-946ac741-a674-49bc-95ac-dc933f64eba2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22dfb3c6720b4eae93360a4a544f5eac', 'request_id': 'req-946ac741-a674-49bc-95ac-dc933f64eba2'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:27.645537 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-946ac741-a674-49bc-95ac-dc933f64eba2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22dfb3c6720b4eae93360a4a544f5eac', 'request_id': 'req-946ac741-a674-49bc-95ac-dc933f64eba2'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:27.734116 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-946ac741-a674-49bc-95ac-dc933f64eba2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22dfb3c6720b4eae93360a4a544f5eac', 'request_id': 'req-946ac741-a674-49bc-95ac-dc933f64eba2'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:27.744568 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-946ac741-a674-49bc-95ac-dc933f64eba2 admin admin] The token cache is being invalidate because project 22dfb3c6720b4eae93360a4a544f5eac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:04:27.744858 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-946ac741-a674-49bc-95ac-dc933f64eba2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:27.745502 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-946ac741-a674-49bc-95ac-dc933f64eba2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:27.747024 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1407/2826] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 06:04:27 2022] DELETE /identity/v3/projects/22dfb3c6720b4eae93360a4a544f5eac => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:04:27.752053 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17da2b2e-893a-4b1b-be1f-e9fe62b9bf01 service neutron] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:27.761677 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17da2b2e-893a-4b1b-be1f-e9fe62b9bf01 service neutron] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:27.817680 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c29da-d95b-4816-b5af-a511b3dc98ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:27.818301 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c29da-d95b-4816-b5af-a511b3dc98ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:27.818581 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c29da-d95b-4816-b5af-a511b3dc98ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:27.818856 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c29da-d95b-4816-b5af-a511b3dc98ea admin admin] PATH_INFO: `/v3/users/d089e03539da47a29c2bba195d2c8105` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:27.819988 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3c29da-d95b-4816-b5af-a511b3dc98ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=d089e03539da47a29c2bba195d2c8105)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:27.821956 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3c29da-d95b-4816-b5af-a511b3dc98ea admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:28.028438 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-ce3c29da-d95b-4816-b5af-a511b3dc98ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd089e03539da47a29c2bba195d2c8105'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:28.141131 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-ce3c29da-d95b-4816-b5af-a511b3dc98ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd089e03539da47a29c2bba195d2c8105'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:28.146149 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:04:28.149156 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-ce3c29da-d95b-4816-b5af-a511b3dc98ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd089e03539da47a29c2bba195d2c8105'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:28.161396 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-ce3c29da-d95b-4816-b5af-a511b3dc98ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd089e03539da47a29c2bba195d2c8105'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:28.162123 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-ce3c29da-d95b-4816-b5af-a511b3dc98ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd089e03539da47a29c2bba195d2c8105'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:28.168133 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1420/2827] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 06:04:27 2022] DELETE /identity/v3/users/d089e03539da47a29c2bba195d2c8105 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:04:28.576859 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946ac741-a674-49bc-95ac-dc933f64eba2 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:28.579040 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-946ac741-a674-49bc-95ac-dc933f64eba2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:28.592701 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946ac741-a674-49bc-95ac-dc933f64eba2 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:28.649944 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823bb343-13a8-42d4-8793-3a4c8d9ecedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:28.650574 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823bb343-13a8-42d4-8793-3a4c8d9ecedb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:28.650868 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823bb343-13a8-42d4-8793-3a4c8d9ecedb admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:28.651127 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823bb343-13a8-42d4-8793-3a4c8d9ecedb admin admin] PATH_INFO: `/v3/projects/b5bf0e35ed414f8e9c36191994188fcb` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:28.652241 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823bb343-13a8-42d4-8793-3a4c8d9ecedb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5bf0e35ed414f8e9c36191994188fcb)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:28.654182 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823bb343-13a8-42d4-8793-3a4c8d9ecedb admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:28.750505 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-823bb343-13a8-42d4-8793-3a4c8d9ecedb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5bf0e35ed414f8e9c36191994188fcb', 'request_id': 'req-823bb343-13a8-42d4-8793-3a4c8d9ecedb'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:28.756765 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-823bb343-13a8-42d4-8793-3a4c8d9ecedb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5bf0e35ed414f8e9c36191994188fcb', 'request_id': 'req-823bb343-13a8-42d4-8793-3a4c8d9ecedb'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:28.849183 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-823bb343-13a8-42d4-8793-3a4c8d9ecedb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5bf0e35ed414f8e9c36191994188fcb', 'request_id': 'req-823bb343-13a8-42d4-8793-3a4c8d9ecedb'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:28.860246 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-823bb343-13a8-42d4-8793-3a4c8d9ecedb admin admin] The token cache is being invalidate because project b5bf0e35ed414f8e9c36191994188fcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:04:28.860582 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-823bb343-13a8-42d4-8793-3a4c8d9ecedb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:28.861297 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-823bb343-13a8-42d4-8793-3a4c8d9ecedb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:28.862817 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1408/2828] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 06:04:28 2022] DELETE /identity/v3/projects/b5bf0e35ed414f8e9c36191994188fcb => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:04:28.867815 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc2a1f-6eff-48b3-9587-cfeceeb31447 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:28.868301 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc2a1f-6eff-48b3-9587-cfeceeb31447 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:28.868435 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc2a1f-6eff-48b3-9587-cfeceeb31447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:28.888034 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-94cc2a1f-6eff-48b3-9587-cfeceeb31447 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:04:28.889255 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-94cc2a1f-6eff-48b3-9587-cfeceeb31447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:28.947684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1421/2829] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:04:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:04:28.955459 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a018dc-4bfe-46a8-b16d-f518d4f28b9a None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:28.955743 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a018dc-4bfe-46a8-b16d-f518d4f28b9a None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:28.956017 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a018dc-4bfe-46a8-b16d-f518d4f28b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:28.976050 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-58a018dc-4bfe-46a8-b16d-f518d4f28b9a None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:04:28.977262 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-58a018dc-4bfe-46a8-b16d-f518d4f28b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:28.986768 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1409/2830] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:04:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:04:28.991083 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc2a1f-6eff-48b3-9587-cfeceeb31447 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:29.001082 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc2a1f-6eff-48b3-9587-cfeceeb31447 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:29.005751 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7deb6ca-2685-41d0-892f-7479270228ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:29.006346 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7deb6ca-2685-41d0-892f-7479270228ed admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:29.006630 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7deb6ca-2685-41d0-892f-7479270228ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:29.006906 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7deb6ca-2685-41d0-892f-7479270228ed admin admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:29.007549 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7deb6ca-2685-41d0-892f-7479270228ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:04:29.007855 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7deb6ca-2685-41d0-892f-7479270228ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:29.009496 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7deb6ca-2685-41d0-892f-7479270228ed admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:29.020985 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1422/2831] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 06:04:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:04:29.025243 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a018dc-4bfe-46a8-b16d-f518d4f28b9a None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:29.035677 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a018dc-4bfe-46a8-b16d-f518d4f28b9a None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:29.040550 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196471c0-cd8f-4d63-b694-2eb6082e2555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:29.041149 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196471c0-cd8f-4d63-b694-2eb6082e2555 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:29.041433 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196471c0-cd8f-4d63-b694-2eb6082e2555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:29.041712 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196471c0-cd8f-4d63-b694-2eb6082e2555 admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:29.042407 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196471c0-cd8f-4d63-b694-2eb6082e2555 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:29.044311 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196471c0-cd8f-4d63-b694-2eb6082e2555 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:29.049996 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:04:29.158896 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1410/2832] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 06:04:29 2022] POST /identity/v3/projects => generated 393 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:04:29.163118 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7deb6ca-2685-41d0-892f-7479270228ed admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:29.173259 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7deb6ca-2685-41d0-892f-7479270228ed admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:29.237326 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7395c73-2f8c-407f-a289-3faa2e9599bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:29.237940 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7395c73-2f8c-407f-a289-3faa2e9599bb admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:29.238228 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7395c73-2f8c-407f-a289-3faa2e9599bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:29.238510 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7395c73-2f8c-407f-a289-3faa2e9599bb admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:29.239148 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7395c73-2f8c-407f-a289-3faa2e9599bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:29.241120 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7395c73-2f8c-407f-a289-3faa2e9599bb admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:29.341199 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1423/2833] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 06:04:29 2022] POST /identity/v3/users => generated 297 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:04:29.345184 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196471c0-cd8f-4d63-b694-2eb6082e2555 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:29.355440 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196471c0-cd8f-4d63-b694-2eb6082e2555 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:29.360601 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a904eb11-82ad-4cfd-a0fe-b7d6bb9d165a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:29.361192 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904eb11-82ad-4cfd-a0fe-b7d6bb9d165a admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:29.361470 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904eb11-82ad-4cfd-a0fe-b7d6bb9d165a admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:29.361741 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904eb11-82ad-4cfd-a0fe-b7d6bb9d165a admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:29.362444 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a904eb11-82ad-4cfd-a0fe-b7d6bb9d165a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:29.364018 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a904eb11-82ad-4cfd-a0fe-b7d6bb9d165a admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:29.373788 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1411/2834] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 06:04:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:04:29.378557 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7395c73-2f8c-407f-a289-3faa2e9599bb admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:29.388485 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7395c73-2f8c-407f-a289-3faa2e9599bb admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:29.393146 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734aefa7-7cb2-4205-b8f2-778510488588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:29.393725 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734aefa7-7cb2-4205-b8f2-778510488588 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:29.394005 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734aefa7-7cb2-4205-b8f2-778510488588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:29.394281 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734aefa7-7cb2-4205-b8f2-778510488588 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:29.394917 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734aefa7-7cb2-4205-b8f2-778510488588 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:29.396554 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734aefa7-7cb2-4205-b8f2-778510488588 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:29.406776 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1424/2835] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 06:04:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:04:29.410809 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a904eb11-82ad-4cfd-a0fe-b7d6bb9d165a admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:29.420524 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a904eb11-82ad-4cfd-a0fe-b7d6bb9d165a admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:29.425561 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b503f565-6558-4631-ae89-240d4b1218d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:29.426196 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503f565-6558-4631-ae89-240d4b1218d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:29.426475 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503f565-6558-4631-ae89-240d4b1218d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:29.426747 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503f565-6558-4631-ae89-240d4b1218d1 admin admin] PATH_INFO: `/v3/projects/a0e890fc35584f7fbed3ae8c78a4cab0/users/38fcf8a7646642b8be8fff4b4027de02/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:29.441935 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b503f565-6558-4631-ae89-240d4b1218d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0e890fc35584f7fbed3ae8c78a4cab0, user_id=38fcf8a7646642b8be8fff4b4027de02, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:29.444615 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b503f565-6558-4631-ae89-240d4b1218d1 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:29.541384 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1412/2836] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 06:04:29 2022] PUT /identity/v3/projects/a0e890fc35584f7fbed3ae8c78a4cab0/users/38fcf8a7646642b8be8fff4b4027de02/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:04:29.553053 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338db999-c40f-4659-8ffb-251f92ee9a2f None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:29.553480 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338db999-c40f-4659-8ffb-251f92ee9a2f None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:29.553974 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338db999-c40f-4659-8ffb-251f92ee9a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:29.582737 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-338db999-c40f-4659-8ffb-251f92ee9a2f None None] MFA Rules not processed for user `38fcf8a7646642b8be8fff4b4027de02`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:04:29.583956 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-338db999-c40f-4659-8ffb-251f92ee9a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:29.613266 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:04:29.648328 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1425/2837] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:04:29 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:04:29.652847 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b503f565-6558-4631-ae89-240d4b1218d1 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:29.715324 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b503f565-6558-4631-ae89-240d4b1218d1 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:29.720584 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff2293-c21b-4844-b08a-b778828cb82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:29.721227 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff2293-c21b-4844-b08a-b778828cb82e admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:29.721506 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff2293-c21b-4844-b08a-b778828cb82e admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:29.721779 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff2293-c21b-4844-b08a-b778828cb82e admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:29.722403 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbff2293-c21b-4844-b08a-b778828cb82e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:29.724297 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbff2293-c21b-4844-b08a-b778828cb82e admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:29.805405 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1413/2838] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 06:04:29 2022] POST /identity/v3/projects => generated 393 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:04:29.809549 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338db999-c40f-4659-8ffb-251f92ee9a2f None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:29.820737 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338db999-c40f-4659-8ffb-251f92ee9a2f None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:29.884774 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258d9b0c-06e8-4ca3-84f2-fcd8716b820a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:29.885699 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d9b0c-06e8-4ca3-84f2-fcd8716b820a admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:29.885992 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d9b0c-06e8-4ca3-84f2-fcd8716b820a admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:29.886272 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d9b0c-06e8-4ca3-84f2-fcd8716b820a admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:29.886895 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258d9b0c-06e8-4ca3-84f2-fcd8716b820a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:29.888829 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258d9b0c-06e8-4ca3-84f2-fcd8716b820a admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:29.975550 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1426/2839] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 06:04:29 2022] POST /identity/v3/users => generated 303 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:04:29.979476 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff2293-c21b-4844-b08a-b778828cb82e admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:29.991298 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff2293-c21b-4844-b08a-b778828cb82e admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:29.996320 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e76b56-e77d-4241-a6ba-468b917467a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:29.996964 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e76b56-e77d-4241-a6ba-468b917467a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:29.997307 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e76b56-e77d-4241-a6ba-468b917467a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:29.997562 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e76b56-e77d-4241-a6ba-468b917467a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:29.998209 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e76b56-e77d-4241-a6ba-468b917467a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:29.999817 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e76b56-e77d-4241-a6ba-468b917467a3 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:30.010719 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1414/2840] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 06:04:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:04:30.014966 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258d9b0c-06e8-4ca3-84f2-fcd8716b820a admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:30.025550 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258d9b0c-06e8-4ca3-84f2-fcd8716b820a admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:30.030194 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831f371-9bf7-46c1-92f0-68276b0ce2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:30.030842 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831f371-9bf7-46c1-92f0-68276b0ce2d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:30.031193 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831f371-9bf7-46c1-92f0-68276b0ce2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:30.031435 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831f371-9bf7-46c1-92f0-68276b0ce2d8 admin admin] PATH_INFO: `/v3/projects/f716953c938c45b092d4e3e3b5671560/users/5e4811fd31d94b548e8e636566883013/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:30.046218 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f831f371-9bf7-46c1-92f0-68276b0ce2d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f716953c938c45b092d4e3e3b5671560, user_id=5e4811fd31d94b548e8e636566883013, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:30.049827 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f831f371-9bf7-46c1-92f0-68276b0ce2d8 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:30.147287 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1427/2841] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 06:04:30 2022] PUT /identity/v3/projects/f716953c938c45b092d4e3e3b5671560/users/5e4811fd31d94b548e8e636566883013/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:04:30.151327 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e76b56-e77d-4241-a6ba-468b917467a3 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:30.162748 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e76b56-e77d-4241-a6ba-468b917467a3 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:30.221251 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a1d282-c6b2-4807-b71b-375490f742ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:30.221860 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1d282-c6b2-4807-b71b-375490f742ad admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:30.222142 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1d282-c6b2-4807-b71b-375490f742ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:30.222416 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1d282-c6b2-4807-b71b-375490f742ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:30.223052 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a1d282-c6b2-4807-b71b-375490f742ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:30.224673 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a1d282-c6b2-4807-b71b-375490f742ad admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:30.234239 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1415/2842] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 06:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:04:30.238925 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831f371-9bf7-46c1-92f0-68276b0ce2d8 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:30.249017 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831f371-9bf7-46c1-92f0-68276b0ce2d8 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:30.253702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f09f31-5302-4446-bce5-5ce56d4261ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:30.254346 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f09f31-5302-4446-bce5-5ce56d4261ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:30.254630 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f09f31-5302-4446-bce5-5ce56d4261ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:30.254907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f09f31-5302-4446-bce5-5ce56d4261ba admin admin] PATH_INFO: `/v3/projects/f716953c938c45b092d4e3e3b5671560/users/5e4811fd31d94b548e8e636566883013/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:30.256888 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f09f31-5302-4446-bce5-5ce56d4261ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=f716953c938c45b092d4e3e3b5671560, user_id=5e4811fd31d94b548e8e636566883013, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:30.259522 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f09f31-5302-4446-bce5-5ce56d4261ba admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:30.269899 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1428/2843] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 06:04:30 2022] PUT /identity/v3/projects/f716953c938c45b092d4e3e3b5671560/users/5e4811fd31d94b548e8e636566883013/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:04:30.280974 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d172bf-25e6-46e4-bfdc-6b7bdf093216 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:30.281265 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d172bf-25e6-46e4-bfdc-6b7bdf093216 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:30.281544 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d172bf-25e6-46e4-bfdc-6b7bdf093216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:30.310781 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-17d172bf-25e6-46e4-bfdc-6b7bdf093216 None None] MFA Rules not processed for user `5e4811fd31d94b548e8e636566883013`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:04:30.312005 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-17d172bf-25e6-46e4-bfdc-6b7bdf093216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:30.320087 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:04:30.378647 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1416/2844] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:04:30 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:04:30.389230 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f09f31-5302-4446-bce5-5ce56d4261ba admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:30.390887 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-e6f09f31-5302-4446-bce5-5ce56d4261ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:30.447829 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f09f31-5302-4446-bce5-5ce56d4261ba admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:30.451461 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eace2776-beab-4dc4-a7f7-b313eeb33509 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:30.452056 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eace2776-beab-4dc4-a7f7-b313eeb33509 service cinder] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:30.452333 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eace2776-beab-4dc4-a7f7-b313eeb33509 service cinder] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:30.452650 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eace2776-beab-4dc4-a7f7-b313eeb33509 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:30.462111 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eace2776-beab-4dc4-a7f7-b313eeb33509 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:30.463532 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eace2776-beab-4dc4-a7f7-b313eeb33509 service cinder] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:30.473291 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1429/2845] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:04:30 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:04:30.519263 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d172bf-25e6-46e4-bfdc-6b7bdf093216 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:30.524553 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-17d172bf-25e6-46e4-bfdc-6b7bdf093216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:30.533438 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d172bf-25e6-46e4-bfdc-6b7bdf093216 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:30.537214 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0f11b-9c08-45e1-9ffd-f8b2d60b4939 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:30.537816 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0f11b-9c08-45e1-9ffd-f8b2d60b4939 service cinder] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:30.538100 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0f11b-9c08-45e1-9ffd-f8b2d60b4939 service cinder] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:30.538378 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0f11b-9c08-45e1-9ffd-f8b2d60b4939 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:30.596958 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb0f11b-9c08-45e1-9ffd-f8b2d60b4939 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:30.598376 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb0f11b-9c08-45e1-9ffd-f8b2d60b4939 service cinder] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:30.607037 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1417/2846] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:04:30 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:04:36.768382 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eace2776-beab-4dc4-a7f7-b313eeb33509 service cinder] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:36.799008 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:04:36.831869 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eace2776-beab-4dc4-a7f7-b313eeb33509 service cinder] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:36.837549 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be53c6-b2cc-44fc-a14d-ebceb0ddb964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:36.838219 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be53c6-b2cc-44fc-a14d-ebceb0ddb964 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:36.838710 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be53c6-b2cc-44fc-a14d-ebceb0ddb964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:36.839002 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be53c6-b2cc-44fc-a14d-ebceb0ddb964 admin admin] PATH_INFO: `/v3/users/38fcf8a7646642b8be8fff4b4027de02` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:36.840476 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0be53c6-b2cc-44fc-a14d-ebceb0ddb964 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38fcf8a7646642b8be8fff4b4027de02)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:36.842483 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0be53c6-b2cc-44fc-a14d-ebceb0ddb964 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:37.069187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-c0be53c6-b2cc-44fc-a14d-ebceb0ddb964 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38fcf8a7646642b8be8fff4b4027de02'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:37.146628 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-c0be53c6-b2cc-44fc-a14d-ebceb0ddb964 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38fcf8a7646642b8be8fff4b4027de02'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:37.153980 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-c0be53c6-b2cc-44fc-a14d-ebceb0ddb964 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38fcf8a7646642b8be8fff4b4027de02'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:37.166123 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-c0be53c6-b2cc-44fc-a14d-ebceb0ddb964 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38fcf8a7646642b8be8fff4b4027de02'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:37.166827 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-c0be53c6-b2cc-44fc-a14d-ebceb0ddb964 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38fcf8a7646642b8be8fff4b4027de02'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:37.172780 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1430/2847] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 06:04:36 2022] DELETE /identity/v3/users/38fcf8a7646642b8be8fff4b4027de02 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:04:37.177929 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7e613-82d7-4fd6-b718-75bd3b1e585f None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:37.178471 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7e613-82d7-4fd6-b718-75bd3b1e585f None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:37.179042 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7e613-82d7-4fd6-b718-75bd3b1e585f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:37.198722 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-d2d7e613-82d7-4fd6-b718-75bd3b1e585f None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:04:37.199918 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-d2d7e613-82d7-4fd6-b718-75bd3b1e585f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:37.257487 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1418/2848] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:04:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:04:37.267037 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be53c6-b2cc-44fc-a14d-ebceb0ddb964 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:37.269143 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-c0be53c6-b2cc-44fc-a14d-ebceb0ddb964 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:37.324779 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be53c6-b2cc-44fc-a14d-ebceb0ddb964 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:37.328304 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e648545-ee13-4c6c-a675-6611f92dd519 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:37.328938 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e648545-ee13-4c6c-a675-6611f92dd519 service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:37.329222 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e648545-ee13-4c6c-a675-6611f92dd519 service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:37.329496 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e648545-ee13-4c6c-a675-6611f92dd519 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:37.338473 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e648545-ee13-4c6c-a675-6611f92dd519 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:37.339916 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e648545-ee13-4c6c-a675-6611f92dd519 service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:37.349584 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1431/2849] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:04:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:04:37.788652 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d7e613-82d7-4fd6-b718-75bd3b1e585f None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:37.790174 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-d2d7e613-82d7-4fd6-b718-75bd3b1e585f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:37.804181 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d7e613-82d7-4fd6-b718-75bd3b1e585f None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:37.809386 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7599f0-41f7-496e-b49e-1f86f00df203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:37.810264 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7599f0-41f7-496e-b49e-1f86f00df203 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:37.810549 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7599f0-41f7-496e-b49e-1f86f00df203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:37.810830 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7599f0-41f7-496e-b49e-1f86f00df203 admin admin] PATH_INFO: `/v3/projects/a0e890fc35584f7fbed3ae8c78a4cab0` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:37.812025 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7599f0-41f7-496e-b49e-1f86f00df203 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0e890fc35584f7fbed3ae8c78a4cab0)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:37.814090 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7599f0-41f7-496e-b49e-1f86f00df203 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:37.963658 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:04:37.970412 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-2c7599f0-41f7-496e-b49e-1f86f00df203 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0e890fc35584f7fbed3ae8c78a4cab0', 'request_id': 'req-2c7599f0-41f7-496e-b49e-1f86f00df203'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:37.977272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-2c7599f0-41f7-496e-b49e-1f86f00df203 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0e890fc35584f7fbed3ae8c78a4cab0', 'request_id': 'req-2c7599f0-41f7-496e-b49e-1f86f00df203'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:38.049525 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-2c7599f0-41f7-496e-b49e-1f86f00df203 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0e890fc35584f7fbed3ae8c78a4cab0', 'request_id': 'req-2c7599f0-41f7-496e-b49e-1f86f00df203'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:38.061192 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-2c7599f0-41f7-496e-b49e-1f86f00df203 admin admin] The token cache is being invalidate because project a0e890fc35584f7fbed3ae8c78a4cab0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:04:38.061485 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-2c7599f0-41f7-496e-b49e-1f86f00df203 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:38.062545 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-2c7599f0-41f7-496e-b49e-1f86f00df203 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:38.063637 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1419/2850] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 06:04:37 2022] DELETE /identity/v3/projects/a0e890fc35584f7fbed3ae8c78a4cab0 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:04:38.067697 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e648545-ee13-4c6c-a675-6611f92dd519 service neutron] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:38.077371 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e648545-ee13-4c6c-a675-6611f92dd519 service neutron] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:38.122937 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:04:38.134856 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e62839-cce4-4f82-a1e4-89d8dd321d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:38.135480 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e62839-cce4-4f82-a1e4-89d8dd321d5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:38.135758 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e62839-cce4-4f82-a1e4-89d8dd321d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:38.136032 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e62839-cce4-4f82-a1e4-89d8dd321d5f admin admin] PATH_INFO: `/v3/users/5e4811fd31d94b548e8e636566883013` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:38.137206 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e62839-cce4-4f82-a1e4-89d8dd321d5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e4811fd31d94b548e8e636566883013)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:38.139116 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e62839-cce4-4f82-a1e4-89d8dd321d5f admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:38.376552 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-d9e62839-cce4-4f82-a1e4-89d8dd321d5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e4811fd31d94b548e8e636566883013'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:38.564150 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-d9e62839-cce4-4f82-a1e4-89d8dd321d5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e4811fd31d94b548e8e636566883013'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:38.572307 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-d9e62839-cce4-4f82-a1e4-89d8dd321d5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e4811fd31d94b548e8e636566883013'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:38.584392 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-d9e62839-cce4-4f82-a1e4-89d8dd321d5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e4811fd31d94b548e8e636566883013'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:38.585096 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-d9e62839-cce4-4f82-a1e4-89d8dd321d5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e4811fd31d94b548e8e636566883013'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:38.591034 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1432/2851] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 06:04:38 2022] DELETE /identity/v3/users/5e4811fd31d94b548e8e636566883013 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:04:39.057662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7599f0-41f7-496e-b49e-1f86f00df203 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:39.059115 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-2c7599f0-41f7-496e-b49e-1f86f00df203 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:39.072576 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7599f0-41f7-496e-b49e-1f86f00df203 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:39.137999 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac830a3-e801-4b42-8ab4-57880741f95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:39.138629 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac830a3-e801-4b42-8ab4-57880741f95e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:39.138921 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac830a3-e801-4b42-8ab4-57880741f95e admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:39.139201 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac830a3-e801-4b42-8ab4-57880741f95e admin admin] PATH_INFO: `/v3/projects/f716953c938c45b092d4e3e3b5671560` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:39.140326 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac830a3-e801-4b42-8ab4-57880741f95e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f716953c938c45b092d4e3e3b5671560)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:39.142295 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac830a3-e801-4b42-8ab4-57880741f95e admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:39.255852 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-3ac830a3-e801-4b42-8ab4-57880741f95e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f716953c938c45b092d4e3e3b5671560', 'request_id': 'req-3ac830a3-e801-4b42-8ab4-57880741f95e'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:39.261953 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-3ac830a3-e801-4b42-8ab4-57880741f95e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f716953c938c45b092d4e3e3b5671560', 'request_id': 'req-3ac830a3-e801-4b42-8ab4-57880741f95e'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:39.340759 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-3ac830a3-e801-4b42-8ab4-57880741f95e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f716953c938c45b092d4e3e3b5671560', 'request_id': 'req-3ac830a3-e801-4b42-8ab4-57880741f95e'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:39.352937 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-3ac830a3-e801-4b42-8ab4-57880741f95e admin admin] The token cache is being invalidate because project f716953c938c45b092d4e3e3b5671560 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:04:39.353226 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-3ac830a3-e801-4b42-8ab4-57880741f95e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:39.353852 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-3ac830a3-e801-4b42-8ab4-57880741f95e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:39.355360 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1420/2852] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 06:04:39 2022] DELETE /identity/v3/projects/f716953c938c45b092d4e3e3b5671560 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:04:39.360425 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ead5f8-5d3c-4b23-8d6b-0352a1dfb0cd None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:39.360896 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ead5f8-5d3c-4b23-8d6b-0352a1dfb0cd None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:39.361006 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ead5f8-5d3c-4b23-8d6b-0352a1dfb0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:39.381460 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-53ead5f8-5d3c-4b23-8d6b-0352a1dfb0cd None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:04:39.382671 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-53ead5f8-5d3c-4b23-8d6b-0352a1dfb0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:39.443878 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1433/2853] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:04:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:04:39.451534 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a2a655-52a7-48a2-853b-25528b6a5b12 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:39.451853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a2a655-52a7-48a2-853b-25528b6a5b12 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:39.452134 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a2a655-52a7-48a2-853b-25528b6a5b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:39.471898 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-d2a2a655-52a7-48a2-853b-25528b6a5b12 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:04:39.473719 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-d2a2a655-52a7-48a2-853b-25528b6a5b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:39.481853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1421/2854] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:04:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:04:39.486153 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ead5f8-5d3c-4b23-8d6b-0352a1dfb0cd None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:39.495955 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ead5f8-5d3c-4b23-8d6b-0352a1dfb0cd None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:39.500635 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71da67e-8357-485d-b467-172bcb3d18cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:39.501235 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71da67e-8357-485d-b467-172bcb3d18cd admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:39.501517 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71da67e-8357-485d-b467-172bcb3d18cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:39.501792 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71da67e-8357-485d-b467-172bcb3d18cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:39.502442 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71da67e-8357-485d-b467-172bcb3d18cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:04:39.502748 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71da67e-8357-485d-b467-172bcb3d18cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:39.504351 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71da67e-8357-485d-b467-172bcb3d18cd admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:39.515672 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1434/2855] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 06:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:04:39.520420 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a2a655-52a7-48a2-853b-25528b6a5b12 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:39.529648 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a2a655-52a7-48a2-853b-25528b6a5b12 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:39.534415 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f2bb9-e5ad-4ab6-b0ca-1a1ad7f905e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:39.535012 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f2bb9-e5ad-4ab6-b0ca-1a1ad7f905e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:39.535294 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f2bb9-e5ad-4ab6-b0ca-1a1ad7f905e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:39.536173 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f2bb9-e5ad-4ab6-b0ca-1a1ad7f905e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:39.536871 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f2bb9-e5ad-4ab6-b0ca-1a1ad7f905e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:39.538813 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f2bb9-e5ad-4ab6-b0ca-1a1ad7f905e2 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:39.627398 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1422/2856] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 06:04:39 2022] POST /identity/v3/projects => generated 367 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:04:39.632335 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71da67e-8357-485d-b467-172bcb3d18cd admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:39.642202 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71da67e-8357-485d-b467-172bcb3d18cd admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:39.691432 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:04:39.698530 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ac6ef-dd78-4d37-9131-e1916cca0d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:39.699136 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ac6ef-dd78-4d37-9131-e1916cca0d40 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:39.699419 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ac6ef-dd78-4d37-9131-e1916cca0d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:39.699698 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ac6ef-dd78-4d37-9131-e1916cca0d40 admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:39.700330 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24ac6ef-dd78-4d37-9131-e1916cca0d40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:39.702277 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24ac6ef-dd78-4d37-9131-e1916cca0d40 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:39.860562 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1435/2857] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 06:04:39 2022] POST /identity/v3/users => generated 284 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:04:39.864579 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f2bb9-e5ad-4ab6-b0ca-1a1ad7f905e2 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:39.875064 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f2bb9-e5ad-4ab6-b0ca-1a1ad7f905e2 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:39.880181 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f96726c-9e64-49a9-8d2c-867001fea8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:39.880947 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f96726c-9e64-49a9-8d2c-867001fea8d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:39.881230 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f96726c-9e64-49a9-8d2c-867001fea8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:39.881505 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f96726c-9e64-49a9-8d2c-867001fea8d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:39.882147 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f96726c-9e64-49a9-8d2c-867001fea8d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:39.883740 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f96726c-9e64-49a9-8d2c-867001fea8d9 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:39.893536 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1423/2858] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 06:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:04:39.898068 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ac6ef-dd78-4d37-9131-e1916cca0d40 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:39.908027 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ac6ef-dd78-4d37-9131-e1916cca0d40 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:39.912985 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9ea598-f303-439d-8fea-dbc927012b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:39.915090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ea598-f303-439d-8fea-dbc927012b85 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:39.915090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ea598-f303-439d-8fea-dbc927012b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:39.915090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ea598-f303-439d-8fea-dbc927012b85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:39.915090 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9ea598-f303-439d-8fea-dbc927012b85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:39.916367 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9ea598-f303-439d-8fea-dbc927012b85 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:39.926793 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1436/2859] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 06:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:04:39.931165 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f96726c-9e64-49a9-8d2c-867001fea8d9 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:39.941231 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f96726c-9e64-49a9-8d2c-867001fea8d9 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:39.947106 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e91549-325d-48c8-ac61-f9471bd5a0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:39.947790 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e91549-325d-48c8-ac61-f9471bd5a0ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:39.948074 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e91549-325d-48c8-ac61-f9471bd5a0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:39.948351 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e91549-325d-48c8-ac61-f9471bd5a0ea admin admin] PATH_INFO: `/v3/projects/a36cdd6b81324f6fa09ada82d3c1f96d/users/f23a28df00be40a6af071f68223f6b6e/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:39.963584 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e91549-325d-48c8-ac61-f9471bd5a0ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=a36cdd6b81324f6fa09ada82d3c1f96d, user_id=f23a28df00be40a6af071f68223f6b6e, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:39.966331 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e91549-325d-48c8-ac61-f9471bd5a0ea admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:39.968527 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:04:40.049168 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1424/2860] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 06:04:39 2022] PUT /identity/v3/projects/a36cdd6b81324f6fa09ada82d3c1f96d/users/f23a28df00be40a6af071f68223f6b6e/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:04:40.072659 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495c741-f23f-4266-a2b0-eaff505310af None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:40.072944 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495c741-f23f-4266-a2b0-eaff505310af None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:40.073218 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495c741-f23f-4266-a2b0-eaff505310af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:40.100852 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-7495c741-f23f-4266-a2b0-eaff505310af None None] MFA Rules not processed for user `f23a28df00be40a6af071f68223f6b6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:04:40.102075 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-7495c741-f23f-4266-a2b0-eaff505310af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:40.165922 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1437/2861] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:04:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:04:40.170383 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e91549-325d-48c8-ac61-f9471bd5a0ea admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:40.234058 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e91549-325d-48c8-ac61-f9471bd5a0ea admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:40.239281 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff7f30e-c21f-45e5-b601-61387a256ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:40.240113 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff7f30e-c21f-45e5-b601-61387a256ef7 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:40.240419 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff7f30e-c21f-45e5-b601-61387a256ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:40.240701 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff7f30e-c21f-45e5-b601-61387a256ef7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:40.241322 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff7f30e-c21f-45e5-b601-61387a256ef7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:40.243225 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff7f30e-c21f-45e5-b601-61387a256ef7 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:40.365045 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1425/2862] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 06:04:40 2022] POST /identity/v3/projects => generated 365 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:04:40.369052 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495c741-f23f-4266-a2b0-eaff505310af None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:40.380206 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495c741-f23f-4266-a2b0-eaff505310af None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:40.435519 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a6067-c452-4291-b2f7-cef06b35fc24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:40.436051 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a6067-c452-4291-b2f7-cef06b35fc24 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:40.436331 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a6067-c452-4291-b2f7-cef06b35fc24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:40.436636 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a6067-c452-4291-b2f7-cef06b35fc24 admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:40.437270 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4a6067-c452-4291-b2f7-cef06b35fc24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:40.439176 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4a6067-c452-4291-b2f7-cef06b35fc24 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:40.552946 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1438/2863] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 06:04:40 2022] POST /identity/v3/users => generated 289 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:04:40.557382 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff7f30e-c21f-45e5-b601-61387a256ef7 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:40.567948 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff7f30e-c21f-45e5-b601-61387a256ef7 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:40.572582 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc2494-19c7-47ee-ac8e-8272067d127c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:40.573175 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc2494-19c7-47ee-ac8e-8272067d127c admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:40.573460 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc2494-19c7-47ee-ac8e-8272067d127c admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:40.573733 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc2494-19c7-47ee-ac8e-8272067d127c admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:40.574368 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cc2494-19c7-47ee-ac8e-8272067d127c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:40.575958 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cc2494-19c7-47ee-ac8e-8272067d127c admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:40.587684 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1426/2864] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 06:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:04:40.591914 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a6067-c452-4291-b2f7-cef06b35fc24 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:40.602741 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a6067-c452-4291-b2f7-cef06b35fc24 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:40.607381 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8aab41-b242-4585-99d5-42e7a1ebf932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:40.608030 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8aab41-b242-4585-99d5-42e7a1ebf932 admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:40.608313 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8aab41-b242-4585-99d5-42e7a1ebf932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:40.608618 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8aab41-b242-4585-99d5-42e7a1ebf932 admin admin] PATH_INFO: `/v3/projects/79cf111d9c53460ab08870958f95885a/users/b5220083ff7b4e24b0683ac33e7c875d/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:40.623977 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8aab41-b242-4585-99d5-42e7a1ebf932 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79cf111d9c53460ab08870958f95885a, user_id=b5220083ff7b4e24b0683ac33e7c875d, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:40.626731 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8aab41-b242-4585-99d5-42e7a1ebf932 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:40.700420 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1439/2865] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 06:04:40 2022] PUT /identity/v3/projects/79cf111d9c53460ab08870958f95885a/users/b5220083ff7b4e24b0683ac33e7c875d/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:04:40.704374 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc2494-19c7-47ee-ac8e-8272067d127c admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:40.714937 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc2494-19c7-47ee-ac8e-8272067d127c admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:40.772944 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0fa193-29bf-4777-966b-806ad3544ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:40.773543 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0fa193-29bf-4777-966b-806ad3544ad5 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:40.773824 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0fa193-29bf-4777-966b-806ad3544ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:40.774100 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0fa193-29bf-4777-966b-806ad3544ad5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:40.774734 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0fa193-29bf-4777-966b-806ad3544ad5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:40.776321 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0fa193-29bf-4777-966b-806ad3544ad5 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:40.786830 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1427/2866] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 06:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:04:40.790944 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8aab41-b242-4585-99d5-42e7a1ebf932 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:40.801294 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8aab41-b242-4585-99d5-42e7a1ebf932 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:40.806278 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c16f3-63d4-4cac-96ee-f33a340a08fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:40.806931 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c16f3-63d4-4cac-96ee-f33a340a08fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:40.807215 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c16f3-63d4-4cac-96ee-f33a340a08fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:40.807493 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c16f3-63d4-4cac-96ee-f33a340a08fe admin admin] PATH_INFO: `/v3/projects/79cf111d9c53460ab08870958f95885a/users/b5220083ff7b4e24b0683ac33e7c875d/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:40.809439 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13c16f3-63d4-4cac-96ee-f33a340a08fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=79cf111d9c53460ab08870958f95885a, user_id=b5220083ff7b4e24b0683ac33e7c875d, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:40.812129 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13c16f3-63d4-4cac-96ee-f33a340a08fe admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:40.822233 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1440/2867] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 06:04:40 2022] PUT /identity/v3/projects/79cf111d9c53460ab08870958f95885a/users/b5220083ff7b4e24b0683ac33e7c875d/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:04:40.833892 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0afc2-6b88-4900-9602-aa8804be83bc None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:40.834179 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0afc2-6b88-4900-9602-aa8804be83bc None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:40.834461 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0afc2-6b88-4900-9602-aa8804be83bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:40.864010 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-e1c0afc2-6b88-4900-9602-aa8804be83bc None None] MFA Rules not processed for user `b5220083ff7b4e24b0683ac33e7c875d`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:04:40.865427 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-e1c0afc2-6b88-4900-9602-aa8804be83bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:40.915172 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:04:40.934107 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1428/2868] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:04:40 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:04:40.945272 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c16f3-63d4-4cac-96ee-f33a340a08fe admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:40.946954 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-c13c16f3-63d4-4cac-96ee-f33a340a08fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:40.966205 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:04:41.004185 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c16f3-63d4-4cac-96ee-f33a340a08fe admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:41.007943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a804cf2d-0591-4f5e-a09d-c0aacf1309ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:41.008755 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a804cf2d-0591-4f5e-a09d-c0aacf1309ad service cinder] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:41.009206 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a804cf2d-0591-4f5e-a09d-c0aacf1309ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:41.009831 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a804cf2d-0591-4f5e-a09d-c0aacf1309ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:41.018844 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a804cf2d-0591-4f5e-a09d-c0aacf1309ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:41.020842 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a804cf2d-0591-4f5e-a09d-c0aacf1309ad service cinder] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:41.030350 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1441/2869] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:04:40 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:04:41.276708 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c0afc2-6b88-4900-9602-aa8804be83bc None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:41.286497 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c0afc2-6b88-4900-9602-aa8804be83bc None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:41.290055 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699863f4-dd0b-4152-9632-8ab37b5045b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:41.290645 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699863f4-dd0b-4152-9632-8ab37b5045b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:41.290924 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699863f4-dd0b-4152-9632-8ab37b5045b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:41.291199 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699863f4-dd0b-4152-9632-8ab37b5045b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:41.346597 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699863f4-dd0b-4152-9632-8ab37b5045b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:41.348044 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699863f4-dd0b-4152-9632-8ab37b5045b9 service cinder] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:41.357494 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1429/2870] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:04:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:04:48.185562 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a804cf2d-0591-4f5e-a09d-c0aacf1309ad service cinder] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:48.246701 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a804cf2d-0591-4f5e-a09d-c0aacf1309ad service cinder] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:48.251161 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec9fb3-26b2-462e-abd9-ee53f25f53cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:48.251784 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec9fb3-26b2-462e-abd9-ee53f25f53cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:48.252065 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec9fb3-26b2-462e-abd9-ee53f25f53cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:48.252341 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec9fb3-26b2-462e-abd9-ee53f25f53cd admin admin] PATH_INFO: `/v3/users/f23a28df00be40a6af071f68223f6b6e` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:48.253486 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ec9fb3-26b2-462e-abd9-ee53f25f53cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f23a28df00be40a6af071f68223f6b6e)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:48.256076 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ec9fb3-26b2-462e-abd9-ee53f25f53cd admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:48.428389 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-89ec9fb3-26b2-462e-abd9-ee53f25f53cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f23a28df00be40a6af071f68223f6b6e'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:48.500622 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-89ec9fb3-26b2-462e-abd9-ee53f25f53cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f23a28df00be40a6af071f68223f6b6e'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:48.508082 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-89ec9fb3-26b2-462e-abd9-ee53f25f53cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f23a28df00be40a6af071f68223f6b6e'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:48.519952 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-89ec9fb3-26b2-462e-abd9-ee53f25f53cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f23a28df00be40a6af071f68223f6b6e'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:48.522950 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-89ec9fb3-26b2-462e-abd9-ee53f25f53cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f23a28df00be40a6af071f68223f6b6e'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:48.527063 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1442/2871] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 06:04:48 2022] DELETE /identity/v3/users/f23a28df00be40a6af071f68223f6b6e => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:04:48.531687 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc1f09-5961-4207-98f8-92bd0e8c0fc5 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:48.532195 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc1f09-5961-4207-98f8-92bd0e8c0fc5 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:48.532340 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc1f09-5961-4207-98f8-92bd0e8c0fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:48.551874 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-e9bc1f09-5961-4207-98f8-92bd0e8c0fc5 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:04:48.553685 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-e9bc1f09-5961-4207-98f8-92bd0e8c0fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:48.613309 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1430/2872] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:04:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:04:48.623257 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec9fb3-26b2-462e-abd9-ee53f25f53cd admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:48.624781 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-89ec9fb3-26b2-462e-abd9-ee53f25f53cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:48.671853 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:04:48.689671 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec9fb3-26b2-462e-abd9-ee53f25f53cd admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:48.694398 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364db420-afb5-43b6-8945-beea2342cdaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:48.695009 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364db420-afb5-43b6-8945-beea2342cdaf service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:48.695296 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364db420-afb5-43b6-8945-beea2342cdaf service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:48.695593 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364db420-afb5-43b6-8945-beea2342cdaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:48.704418 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364db420-afb5-43b6-8945-beea2342cdaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:48.705927 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364db420-afb5-43b6-8945-beea2342cdaf service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:48.717069 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1443/2873] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:04:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:04:49.201346 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc1f09-5961-4207-98f8-92bd0e8c0fc5 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:49.202938 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-e9bc1f09-5961-4207-98f8-92bd0e8c0fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:49.216256 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc1f09-5961-4207-98f8-92bd0e8c0fc5 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:49.221295 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034effb-f8e6-4f80-b24f-3fd5bf932284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:49.221936 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034effb-f8e6-4f80-b24f-3fd5bf932284 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:49.223098 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034effb-f8e6-4f80-b24f-3fd5bf932284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:49.223782 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034effb-f8e6-4f80-b24f-3fd5bf932284 admin admin] PATH_INFO: `/v3/projects/a36cdd6b81324f6fa09ada82d3c1f96d` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:49.225284 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6034effb-f8e6-4f80-b24f-3fd5bf932284 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a36cdd6b81324f6fa09ada82d3c1f96d)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:49.227210 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6034effb-f8e6-4f80-b24f-3fd5bf932284 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:49.387851 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-6034effb-f8e6-4f80-b24f-3fd5bf932284 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a36cdd6b81324f6fa09ada82d3c1f96d', 'request_id': 'req-6034effb-f8e6-4f80-b24f-3fd5bf932284'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:49.394070 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-6034effb-f8e6-4f80-b24f-3fd5bf932284 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a36cdd6b81324f6fa09ada82d3c1f96d', 'request_id': 'req-6034effb-f8e6-4f80-b24f-3fd5bf932284'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:49.483480 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-6034effb-f8e6-4f80-b24f-3fd5bf932284 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a36cdd6b81324f6fa09ada82d3c1f96d', 'request_id': 'req-6034effb-f8e6-4f80-b24f-3fd5bf932284'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:49.496016 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-6034effb-f8e6-4f80-b24f-3fd5bf932284 admin admin] The token cache is being invalidate because project a36cdd6b81324f6fa09ada82d3c1f96d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:04:49.496310 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-6034effb-f8e6-4f80-b24f-3fd5bf932284 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:49.496978 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-6034effb-f8e6-4f80-b24f-3fd5bf932284 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:49.498485 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1431/2874] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 06:04:49 2022] DELETE /identity/v3/projects/a36cdd6b81324f6fa09ada82d3c1f96d => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:04:49.502488 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364db420-afb5-43b6-8945-beea2342cdaf service neutron] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:49.512211 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364db420-afb5-43b6-8945-beea2342cdaf service neutron] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:49.570509 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9189436-af0e-40fe-bfac-bf74abfbdb73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:49.571141 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9189436-af0e-40fe-bfac-bf74abfbdb73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:49.571421 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9189436-af0e-40fe-bfac-bf74abfbdb73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:49.571696 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9189436-af0e-40fe-bfac-bf74abfbdb73 admin admin] PATH_INFO: `/v3/users/b5220083ff7b4e24b0683ac33e7c875d` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:49.572845 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9189436-af0e-40fe-bfac-bf74abfbdb73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5220083ff7b4e24b0683ac33e7c875d)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:49.574777 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9189436-af0e-40fe-bfac-bf74abfbdb73 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:49.793947 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-c9189436-af0e-40fe-bfac-bf74abfbdb73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5220083ff7b4e24b0683ac33e7c875d'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:49.887979 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-c9189436-af0e-40fe-bfac-bf74abfbdb73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5220083ff7b4e24b0683ac33e7c875d'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:49.895140 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-c9189436-af0e-40fe-bfac-bf74abfbdb73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5220083ff7b4e24b0683ac33e7c875d'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:49.906867 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-c9189436-af0e-40fe-bfac-bf74abfbdb73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5220083ff7b4e24b0683ac33e7c875d'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:49.908038 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-c9189436-af0e-40fe-bfac-bf74abfbdb73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5220083ff7b4e24b0683ac33e7c875d'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:49.913502 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1444/2875] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 06:04:49 2022] DELETE /identity/v3/users/b5220083ff7b4e24b0683ac33e7c875d => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:04:50.319813 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034effb-f8e6-4f80-b24f-3fd5bf932284 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:50.321365 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-6034effb-f8e6-4f80-b24f-3fd5bf932284 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:50.335811 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034effb-f8e6-4f80-b24f-3fd5bf932284 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:50.343076 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:04:50.392312 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb12f894-64f4-473c-aaa2-45db20d68043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:50.392973 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12f894-64f4-473c-aaa2-45db20d68043 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:50.393277 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12f894-64f4-473c-aaa2-45db20d68043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:50.393538 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12f894-64f4-473c-aaa2-45db20d68043 admin admin] PATH_INFO: `/v3/projects/79cf111d9c53460ab08870958f95885a` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:50.394666 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb12f894-64f4-473c-aaa2-45db20d68043 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79cf111d9c53460ab08870958f95885a)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:50.397204 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb12f894-64f4-473c-aaa2-45db20d68043 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:50.559193 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-eb12f894-64f4-473c-aaa2-45db20d68043 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79cf111d9c53460ab08870958f95885a', 'request_id': 'req-eb12f894-64f4-473c-aaa2-45db20d68043'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:50.565355 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-eb12f894-64f4-473c-aaa2-45db20d68043 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79cf111d9c53460ab08870958f95885a', 'request_id': 'req-eb12f894-64f4-473c-aaa2-45db20d68043'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:50.689261 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-eb12f894-64f4-473c-aaa2-45db20d68043 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79cf111d9c53460ab08870958f95885a', 'request_id': 'req-eb12f894-64f4-473c-aaa2-45db20d68043'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:50.700587 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-eb12f894-64f4-473c-aaa2-45db20d68043 admin admin] The token cache is being invalidate because project 79cf111d9c53460ab08870958f95885a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:04:50.700878 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-eb12f894-64f4-473c-aaa2-45db20d68043 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:50.701593 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-eb12f894-64f4-473c-aaa2-45db20d68043 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:50.703073 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1432/2876] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 06:04:50 2022] DELETE /identity/v3/projects/79cf111d9c53460ab08870958f95885a => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:04:50.708155 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d751de62-731e-402f-a8a3-e70e633484f0 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:50.708684 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d751de62-731e-402f-a8a3-e70e633484f0 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:50.709570 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d751de62-731e-402f-a8a3-e70e633484f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:50.729791 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-d751de62-731e-402f-a8a3-e70e633484f0 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:04:50.730991 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-d751de62-731e-402f-a8a3-e70e633484f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:50.780159 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:04:50.788943 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1445/2877] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:04:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:04:50.796742 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89467bda-66c0-40ba-a1d0-e6d318db35b9 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:50.797030 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89467bda-66c0-40ba-a1d0-e6d318db35b9 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:50.797306 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89467bda-66c0-40ba-a1d0-e6d318db35b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:50.817445 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-89467bda-66c0-40ba-a1d0-e6d318db35b9 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:04:50.818689 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-89467bda-66c0-40ba-a1d0-e6d318db35b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:50.827231 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1433/2878] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:04:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:04:50.831377 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d751de62-731e-402f-a8a3-e70e633484f0 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:50.842345 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d751de62-731e-402f-a8a3-e70e633484f0 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:50.847013 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f987877e-f575-4d34-9959-b949d804c9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:50.849421 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987877e-f575-4d34-9959-b949d804c9c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:50.849421 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987877e-f575-4d34-9959-b949d804c9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:50.849421 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987877e-f575-4d34-9959-b949d804c9c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:50.849421 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f987877e-f575-4d34-9959-b949d804c9c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:04:50.849421 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f987877e-f575-4d34-9959-b949d804c9c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:50.851284 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f987877e-f575-4d34-9959-b949d804c9c6 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:50.862873 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1446/2879] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 06:04:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:04:50.867496 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89467bda-66c0-40ba-a1d0-e6d318db35b9 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:50.876716 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89467bda-66c0-40ba-a1d0-e6d318db35b9 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:50.881421 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed65617-8d16-41e9-b29a-1a7b72289e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:50.882663 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed65617-8d16-41e9-b29a-1a7b72289e64 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:50.882887 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed65617-8d16-41e9-b29a-1a7b72289e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:50.883126 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed65617-8d16-41e9-b29a-1a7b72289e64 admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:50.883748 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed65617-8d16-41e9-b29a-1a7b72289e64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:50.885666 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed65617-8d16-41e9-b29a-1a7b72289e64 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:50.965959 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1434/2880] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 06:04:50 2022] POST /identity/v3/projects => generated 373 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:04:50.970093 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f987877e-f575-4d34-9959-b949d804c9c6 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:50.980346 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f987877e-f575-4d34-9959-b949d804c9c6 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:51.036262 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca1b18-20c5-4642-a4eb-fcdad53566b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:51.036930 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca1b18-20c5-4642-a4eb-fcdad53566b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:51.037217 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca1b18-20c5-4642-a4eb-fcdad53566b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:51.037496 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca1b18-20c5-4642-a4eb-fcdad53566b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:51.038228 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca1b18-20c5-4642-a4eb-fcdad53566b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:51.040154 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca1b18-20c5-4642-a4eb-fcdad53566b2 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:51.210701 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1447/2881] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 06:04:50 2022] POST /identity/v3/users => generated 287 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:04:51.214988 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed65617-8d16-41e9-b29a-1a7b72289e64 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:51.225313 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed65617-8d16-41e9-b29a-1a7b72289e64 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:51.229769 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c605dcdd-4bc9-4fb3-a63b-4a4a5b8748a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:51.230361 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605dcdd-4bc9-4fb3-a63b-4a4a5b8748a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:51.230675 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605dcdd-4bc9-4fb3-a63b-4a4a5b8748a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:51.230946 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605dcdd-4bc9-4fb3-a63b-4a4a5b8748a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:51.231579 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c605dcdd-4bc9-4fb3-a63b-4a4a5b8748a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:51.233207 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c605dcdd-4bc9-4fb3-a63b-4a4a5b8748a4 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:51.243912 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1435/2882] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 06:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:04:51.247792 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca1b18-20c5-4642-a4eb-fcdad53566b2 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:51.258511 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca1b18-20c5-4642-a4eb-fcdad53566b2 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:51.263330 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca24699-1fe6-4d05-9390-6ff9cbe7419f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:51.263964 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca24699-1fe6-4d05-9390-6ff9cbe7419f admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:51.264276 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca24699-1fe6-4d05-9390-6ff9cbe7419f admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:51.264705 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca24699-1fe6-4d05-9390-6ff9cbe7419f admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:51.265360 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca24699-1fe6-4d05-9390-6ff9cbe7419f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:51.266958 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca24699-1fe6-4d05-9390-6ff9cbe7419f admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:51.277768 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1448/2883] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 06:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:04:51.281952 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c605dcdd-4bc9-4fb3-a63b-4a4a5b8748a4 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:51.292349 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c605dcdd-4bc9-4fb3-a63b-4a4a5b8748a4 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:51.296924 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2acab4-6639-4fdc-ac09-d592dd5430fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:51.297570 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2acab4-6639-4fdc-ac09-d592dd5430fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:51.297851 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2acab4-6639-4fdc-ac09-d592dd5430fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:51.298131 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2acab4-6639-4fdc-ac09-d592dd5430fa admin admin] PATH_INFO: `/v3/projects/e2851b48f7c6416fa29082ccde5c4a82/users/3df1f4267bc149bcbef4b569f7e67f85/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:51.314444 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2acab4-6639-4fdc-ac09-d592dd5430fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2851b48f7c6416fa29082ccde5c4a82, user_id=3df1f4267bc149bcbef4b569f7e67f85, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:51.317423 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2acab4-6639-4fdc-ac09-d592dd5430fa admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:51.398176 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1436/2884] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 06:04:51 2022] PUT /identity/v3/projects/e2851b48f7c6416fa29082ccde5c4a82/users/3df1f4267bc149bcbef4b569f7e67f85/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:04:51.414545 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b6457-87d1-46ae-acef-2b0b5b57ca2e None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:51.415058 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b6457-87d1-46ae-acef-2b0b5b57ca2e None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:51.415550 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b6457-87d1-46ae-acef-2b0b5b57ca2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:51.444892 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-e32b6457-87d1-46ae-acef-2b0b5b57ca2e None None] MFA Rules not processed for user `3df1f4267bc149bcbef4b569f7e67f85`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:04:51.446288 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-e32b6457-87d1-46ae-acef-2b0b5b57ca2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:51.512127 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1449/2885] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:04:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:04:51.523272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2acab4-6639-4fdc-ac09-d592dd5430fa admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:51.525011 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-9a2acab4-6639-4fdc-ac09-d592dd5430fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:51.567220 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:04:51.582725 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2acab4-6639-4fdc-ac09-d592dd5430fa admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:51.586484 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0b1e81-c8f0-4ed3-878b-c8e2aca577ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:51.587080 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b1e81-c8f0-4ed3-878b-c8e2aca577ad service cinder] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:51.587357 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b1e81-c8f0-4ed3-878b-c8e2aca577ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:51.587629 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b1e81-c8f0-4ed3-878b-c8e2aca577ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:51.595705 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0b1e81-c8f0-4ed3-878b-c8e2aca577ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:51.597154 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0b1e81-c8f0-4ed3-878b-c8e2aca577ad service cinder] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:51.606259 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1437/2886] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:04:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:04:51.637933 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b6457-87d1-46ae-acef-2b0b5b57ca2e None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:51.696148 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:04:51.701516 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b6457-87d1-46ae-acef-2b0b5b57ca2e None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:51.706096 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b46ed-e75e-4504-9f67-bb801b4beaca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:51.706715 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b46ed-e75e-4504-9f67-bb801b4beaca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:51.706994 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b46ed-e75e-4504-9f67-bb801b4beaca admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:51.707266 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b46ed-e75e-4504-9f67-bb801b4beaca admin admin] PATH_INFO: `/v3/users/3df1f4267bc149bcbef4b569f7e67f85` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:51.708391 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535b46ed-e75e-4504-9f67-bb801b4beaca admin admin] RBAC: Authorizing `identity:delete_user(user_id=3df1f4267bc149bcbef4b569f7e67f85)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:51.710459 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535b46ed-e75e-4504-9f67-bb801b4beaca admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:52.020636 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-535b46ed-e75e-4504-9f67-bb801b4beaca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3df1f4267bc149bcbef4b569f7e67f85'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:52.123249 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-535b46ed-e75e-4504-9f67-bb801b4beaca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3df1f4267bc149bcbef4b569f7e67f85'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:52.130789 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-535b46ed-e75e-4504-9f67-bb801b4beaca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3df1f4267bc149bcbef4b569f7e67f85'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:52.143286 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-535b46ed-e75e-4504-9f67-bb801b4beaca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3df1f4267bc149bcbef4b569f7e67f85'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:52.143965 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-535b46ed-e75e-4504-9f67-bb801b4beaca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3df1f4267bc149bcbef4b569f7e67f85'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:52.150894 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1450/2887] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 06:04:51 2022] DELETE /identity/v3/users/3df1f4267bc149bcbef4b569f7e67f85 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:04:52.155320 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b68ff-ac5b-4efc-8c78-93beb0fcc8c0 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:52.155607 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b68ff-ac5b-4efc-8c78-93beb0fcc8c0 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:52.155883 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b68ff-ac5b-4efc-8c78-93beb0fcc8c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:52.175800 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-819b68ff-ac5b-4efc-8c78-93beb0fcc8c0 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:04:52.177048 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-819b68ff-ac5b-4efc-8c78-93beb0fcc8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:52.242671 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1438/2888] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:04:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:04:52.253300 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b46ed-e75e-4504-9f67-bb801b4beaca admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:52.254877 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-535b46ed-e75e-4504-9f67-bb801b4beaca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:52.317636 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b46ed-e75e-4504-9f67-bb801b4beaca admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:52.321907 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d707e90-6a6a-4561-8da5-cebe28fdc07d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:52.322527 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d707e90-6a6a-4561-8da5-cebe28fdc07d service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:52.322811 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d707e90-6a6a-4561-8da5-cebe28fdc07d service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:52.323087 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d707e90-6a6a-4561-8da5-cebe28fdc07d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:52.332977 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d707e90-6a6a-4561-8da5-cebe28fdc07d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:52.334440 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d707e90-6a6a-4561-8da5-cebe28fdc07d service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:52.345359 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1451/2889] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:04:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:04:52.772547 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b68ff-ac5b-4efc-8c78-93beb0fcc8c0 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:52.776882 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-819b68ff-ac5b-4efc-8c78-93beb0fcc8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:52.786147 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b68ff-ac5b-4efc-8c78-93beb0fcc8c0 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:52.790779 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74b4c19-43d7-44f5-83ab-969a0a49a0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:52.791414 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b4c19-43d7-44f5-83ab-969a0a49a0f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:52.791697 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b4c19-43d7-44f5-83ab-969a0a49a0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:52.791973 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b4c19-43d7-44f5-83ab-969a0a49a0f5 admin admin] PATH_INFO: `/v3/projects/e2851b48f7c6416fa29082ccde5c4a82` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:52.793136 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74b4c19-43d7-44f5-83ab-969a0a49a0f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2851b48f7c6416fa29082ccde5c4a82)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:52.795050 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74b4c19-43d7-44f5-83ab-969a0a49a0f5 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:52.933593 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-b74b4c19-43d7-44f5-83ab-969a0a49a0f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2851b48f7c6416fa29082ccde5c4a82', 'request_id': 'req-b74b4c19-43d7-44f5-83ab-969a0a49a0f5'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:52.940521 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-b74b4c19-43d7-44f5-83ab-969a0a49a0f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2851b48f7c6416fa29082ccde5c4a82', 'request_id': 'req-b74b4c19-43d7-44f5-83ab-969a0a49a0f5'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:53.083443 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-b74b4c19-43d7-44f5-83ab-969a0a49a0f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2851b48f7c6416fa29082ccde5c4a82', 'request_id': 'req-b74b4c19-43d7-44f5-83ab-969a0a49a0f5'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:53.094851 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-b74b4c19-43d7-44f5-83ab-969a0a49a0f5 admin admin] The token cache is being invalidate because project e2851b48f7c6416fa29082ccde5c4a82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:04:53.095140 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-b74b4c19-43d7-44f5-83ab-969a0a49a0f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:53.095770 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-b74b4c19-43d7-44f5-83ab-969a0a49a0f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:53.097295 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1439/2890] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 06:04:52 2022] DELETE /identity/v3/projects/e2851b48f7c6416fa29082ccde5c4a82 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:04:53.102764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4c53f-6bd8-4031-9ebf-9dda9158b9f7 None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:53.103213 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4c53f-6bd8-4031-9ebf-9dda9158b9f7 None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:53.103316 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4c53f-6bd8-4031-9ebf-9dda9158b9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:53.123579 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-7fc4c53f-6bd8-4031-9ebf-9dda9158b9f7 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:04:53.124816 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-7fc4c53f-6bd8-4031-9ebf-9dda9158b9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:53.197840 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1452/2891] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:04:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:04:53.205276 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43289858-260f-413a-9a8f-4ab0d370b952 None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:53.205639 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43289858-260f-413a-9a8f-4ab0d370b952 None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:53.205924 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43289858-260f-413a-9a8f-4ab0d370b952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:53.227162 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-43289858-260f-413a-9a8f-4ab0d370b952 None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:04:53.228379 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-43289858-260f-413a-9a8f-4ab0d370b952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:53.237244 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1440/2892] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:04:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:04:53.242857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc4c53f-6bd8-4031-9ebf-9dda9158b9f7 None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:53.253000 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc4c53f-6bd8-4031-9ebf-9dda9158b9f7 None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:53.258752 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9d5049-5435-43a4-99ce-ce0ddc7770db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:53.259364 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d5049-5435-43a4-99ce-ce0ddc7770db admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:53.259648 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d5049-5435-43a4-99ce-ce0ddc7770db admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:53.259926 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d5049-5435-43a4-99ce-ce0ddc7770db admin admin] PATH_INFO: `/v3/domains` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:53.260610 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9d5049-5435-43a4-99ce-ce0ddc7770db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81670) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:04:53.260922 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9d5049-5435-43a4-99ce-ce0ddc7770db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:53.262525 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9d5049-5435-43a4-99ce-ce0ddc7770db admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:53.274884 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1453/2893] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Aug 24 06:04:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:04:53.279188 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43289858-260f-413a-9a8f-4ab0d370b952 None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:53.289699 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43289858-260f-413a-9a8f-4ab0d370b952 None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:53.294687 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ee303f-5986-4f3c-be94-8c54d02a928b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:53.295294 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee303f-5986-4f3c-be94-8c54d02a928b admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:53.295577 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee303f-5986-4f3c-be94-8c54d02a928b admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:53.295853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee303f-5986-4f3c-be94-8c54d02a928b admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:53.296547 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ee303f-5986-4f3c-be94-8c54d02a928b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:53.298481 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ee303f-5986-4f3c-be94-8c54d02a928b admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:53.403519 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1441/2894] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 06:04:53 2022] POST /identity/v3/projects => generated 373 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:04:53.407603 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9d5049-5435-43a4-99ce-ce0ddc7770db admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:53.414423 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:04:53.418186 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9d5049-5435-43a4-99ce-ce0ddc7770db admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:53.478884 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6f29b-f086-4887-b714-4da518d61363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:53.479408 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6f29b-f086-4887-b714-4da518d61363 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:53.479686 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6f29b-f086-4887-b714-4da518d61363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:53.479958 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6f29b-f086-4887-b714-4da518d61363 admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:53.480609 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b6f29b-f086-4887-b714-4da518d61363 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:53.482541 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b6f29b-f086-4887-b714-4da518d61363 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:53.587849 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1454/2895] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 06:04:53 2022] POST /identity/v3/users => generated 287 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:04:53.592127 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ee303f-5986-4f3c-be94-8c54d02a928b admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:53.603330 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ee303f-5986-4f3c-be94-8c54d02a928b admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:53.607986 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876b2e4-2bbe-4a9e-8192-4c65e4c2d7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:53.608615 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876b2e4-2bbe-4a9e-8192-4c65e4c2d7ac admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:53.608899 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876b2e4-2bbe-4a9e-8192-4c65e4c2d7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:53.609176 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876b2e4-2bbe-4a9e-8192-4c65e4c2d7ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:53.609816 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7876b2e4-2bbe-4a9e-8192-4c65e4c2d7ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:53.611413 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7876b2e4-2bbe-4a9e-8192-4c65e4c2d7ac admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:53.621533 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1442/2896] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 06:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:04:53.627315 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6f29b-f086-4887-b714-4da518d61363 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:53.637365 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6f29b-f086-4887-b714-4da518d61363 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:53.642008 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a70c57-5a6a-4517-a3ae-258179e46459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:53.642600 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a70c57-5a6a-4517-a3ae-258179e46459 admin admin] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:53.642880 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a70c57-5a6a-4517-a3ae-258179e46459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:53.643154 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a70c57-5a6a-4517-a3ae-258179e46459 admin admin] PATH_INFO: `/v3/roles` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:53.643795 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a70c57-5a6a-4517-a3ae-258179e46459 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:53.645412 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a70c57-5a6a-4517-a3ae-258179e46459 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:53.656385 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1455/2897] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 06:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:04:53.671559 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876b2e4-2bbe-4a9e-8192-4c65e4c2d7ac admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:53.681968 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876b2e4-2bbe-4a9e-8192-4c65e4c2d7ac admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:53.686587 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e7e61e-0089-468d-8741-c3f13bedb565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:53.687231 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7e61e-0089-468d-8741-c3f13bedb565 admin admin] REQUEST_METHOD: `PUT` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:53.687513 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7e61e-0089-468d-8741-c3f13bedb565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:53.687794 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7e61e-0089-468d-8741-c3f13bedb565 admin admin] PATH_INFO: `/v3/projects/6a56c0475f3242d6aa22227c5c6d84ac/users/a65be4eeb4054d79ab392bc723323339/roles/87862cd6eaeb4d5fae089a801aac4f87` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:53.702588 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e7e61e-0089-468d-8741-c3f13bedb565 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a56c0475f3242d6aa22227c5c6d84ac, user_id=a65be4eeb4054d79ab392bc723323339, role_id=87862cd6eaeb4d5fae089a801aac4f87)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:53.705272 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e7e61e-0089-468d-8741-c3f13bedb565 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:53.787851 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:04:53.792904 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1443/2898] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 06:04:53 2022] PUT /identity/v3/projects/6a56c0475f3242d6aa22227c5c6d84ac/users/a65be4eeb4054d79ab392bc723323339/roles/87862cd6eaeb4d5fae089a801aac4f87 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:04:53.803161 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb46025-1548-4834-a230-71b8c2f8713d None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:53.803466 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb46025-1548-4834-a230-71b8c2f8713d None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:53.803702 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb46025-1548-4834-a230-71b8c2f8713d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:53.831965 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-dbb46025-1548-4834-a230-71b8c2f8713d None None] MFA Rules not processed for user `a65be4eeb4054d79ab392bc723323339`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:04:53.833213 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-dbb46025-1548-4834-a230-71b8c2f8713d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:53.898191 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1456/2899] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:04:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:04:53.902755 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e7e61e-0089-468d-8741-c3f13bedb565 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:53.964534 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e7e61e-0089-468d-8741-c3f13bedb565 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:53.969226 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b44f7-9795-419b-8cfd-cc34d8e2e7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:53.969819 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b44f7-9795-419b-8cfd-cc34d8e2e7c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:53.970099 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b44f7-9795-419b-8cfd-cc34d8e2e7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:53.970376 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b44f7-9795-419b-8cfd-cc34d8e2e7c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:53.970990 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654b44f7-9795-419b-8cfd-cc34d8e2e7c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:53.972939 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654b44f7-9795-419b-8cfd-cc34d8e2e7c6 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:54.056130 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1444/2900] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Aug 24 06:04:53 2022] POST /identity/v3/projects => generated 375 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:04:54.060337 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb46025-1548-4834-a230-71b8c2f8713d None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:54.071567 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb46025-1548-4834-a230-71b8c2f8713d None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:54.127802 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346cfb30-b75e-41e2-b06c-4b7d8cff6eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:54.128475 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346cfb30-b75e-41e2-b06c-4b7d8cff6eb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:54.128758 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346cfb30-b75e-41e2-b06c-4b7d8cff6eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:54.129032 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346cfb30-b75e-41e2-b06c-4b7d8cff6eb0 admin admin] PATH_INFO: `/v3/users` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:54.129664 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346cfb30-b75e-41e2-b06c-4b7d8cff6eb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:54.132082 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346cfb30-b75e-41e2-b06c-4b7d8cff6eb0 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:54.256097 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1457/2901] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Aug 24 06:04:54 2022] POST /identity/v3/users => generated 294 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:04:54.260362 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b44f7-9795-419b-8cfd-cc34d8e2e7c6 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:54.270632 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b44f7-9795-419b-8cfd-cc34d8e2e7c6 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:54.275206 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1501f-f07f-41f8-84b7-08a9020683f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:54.275802 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1501f-f07f-41f8-84b7-08a9020683f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:54.276086 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1501f-f07f-41f8-84b7-08a9020683f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:54.276359 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1501f-f07f-41f8-84b7-08a9020683f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:54.277020 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc1501f-f07f-41f8-84b7-08a9020683f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:54.278632 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc1501f-f07f-41f8-84b7-08a9020683f6 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:54.288911 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1445/2902] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 06:04:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:04:54.292995 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346cfb30-b75e-41e2-b06c-4b7d8cff6eb0 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:54.298221 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:04:54.304223 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346cfb30-b75e-41e2-b06c-4b7d8cff6eb0 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:54.309032 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af70d406-ff10-4470-ac95-98891530e6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:54.309621 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70d406-ff10-4470-ac95-98891530e6aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:54.309903 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70d406-ff10-4470-ac95-98891530e6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:54.310184 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70d406-ff10-4470-ac95-98891530e6aa admin admin] PATH_INFO: `/v3/projects/67e20a540d41490a85c35d05fdd1c202/users/a4e0eb045b684e8c97e35210891b6b15/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:54.325092 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af70d406-ff10-4470-ac95-98891530e6aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=67e20a540d41490a85c35d05fdd1c202, user_id=a4e0eb045b684e8c97e35210891b6b15, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:54.327760 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af70d406-ff10-4470-ac95-98891530e6aa admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:54.454637 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1458/2903] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 06:04:54 2022] PUT /identity/v3/projects/67e20a540d41490a85c35d05fdd1c202/users/a4e0eb045b684e8c97e35210891b6b15/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:04:54.458631 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1501f-f07f-41f8-84b7-08a9020683f6 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:54.468940 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1501f-f07f-41f8-84b7-08a9020683f6 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:54.524990 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b6371b-d57c-43e7-85e3-7fe51281a899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:54.525597 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6371b-d57c-43e7-85e3-7fe51281a899 admin admin] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:54.525880 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6371b-d57c-43e7-85e3-7fe51281a899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:54.526158 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6371b-d57c-43e7-85e3-7fe51281a899 admin admin] PATH_INFO: `/v3/roles` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:54.526798 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b6371b-d57c-43e7-85e3-7fe51281a899 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:54.528419 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b6371b-d57c-43e7-85e3-7fe51281a899 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:54.537903 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1446/2904] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Aug 24 06:04:54 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:04:54.542006 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af70d406-ff10-4470-ac95-98891530e6aa admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:54.552341 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af70d406-ff10-4470-ac95-98891530e6aa admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:54.557874 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7879891e-077a-4d21-93cd-c0a49ddd803c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:54.558560 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7879891e-077a-4d21-93cd-c0a49ddd803c admin admin] REQUEST_METHOD: `PUT` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:54.558838 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7879891e-077a-4d21-93cd-c0a49ddd803c admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:54.559168 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7879891e-077a-4d21-93cd-c0a49ddd803c admin admin] PATH_INFO: `/v3/projects/67e20a540d41490a85c35d05fdd1c202/users/a4e0eb045b684e8c97e35210891b6b15/roles/e013ad4d436f4cb0a9bf7debb7d13dc7` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:54.561126 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7879891e-077a-4d21-93cd-c0a49ddd803c admin admin] RBAC: Authorizing `identity:create_grant(project_id=67e20a540d41490a85c35d05fdd1c202, user_id=a4e0eb045b684e8c97e35210891b6b15, role_id=e013ad4d436f4cb0a9bf7debb7d13dc7)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:54.563864 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7879891e-077a-4d21-93cd-c0a49ddd803c admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:54.575544 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1459/2905] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Aug 24 06:04:54 2022] PUT /identity/v3/projects/67e20a540d41490a85c35d05fdd1c202/users/a4e0eb045b684e8c97e35210891b6b15/roles/e013ad4d436f4cb0a9bf7debb7d13dc7 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:04:54.587140 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee593d-8954-4cb7-87a8-647a133592cb None None] REQUEST_METHOD: `POST` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:54.587515 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee593d-8954-4cb7-87a8-647a133592cb None None] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:54.587855 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee593d-8954-4cb7-87a8-647a133592cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:54.617109 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.auth.core [None req-b7ee593d-8954-4cb7-87a8-647a133592cb None None] MFA Rules not processed for user `a4e0eb045b684e8c97e35210891b6b15`. Rule list: `[]` (Enabled: `True`). {{(pid=81671) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:04:54.618361 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-b7ee593d-8954-4cb7-87a8-647a133592cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:54.680295 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:04:54.693662 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1447/2906] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:04:54 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:04:54.703838 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7879891e-077a-4d21-93cd-c0a49ddd803c admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:54.706187 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-7879891e-077a-4d21-93cd-c0a49ddd803c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:54.766162 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7879891e-077a-4d21-93cd-c0a49ddd803c admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:54.770220 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f21931-77cc-44ad-a9f0-66227f3ab5ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bfd618ef39498f91fe7731cd40253b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:54.770826 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f21931-77cc-44ad-a9f0-66227f3ab5ad service cinder] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:54.771106 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f21931-77cc-44ad-a9f0-66227f3ab5ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:54.771383 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f21931-77cc-44ad-a9f0-66227f3ab5ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:54.836718 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f21931-77cc-44ad-a9f0-66227f3ab5ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:54.836718 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f21931-77cc-44ad-a9f0-66227f3ab5ad service cinder] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:54.842047 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1460/2907] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:04:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:04:58.213254 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ee593d-8954-4cb7-87a8-647a133592cb None None] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:58.273450 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ee593d-8954-4cb7-87a8-647a133592cb None None] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:58.278116 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc109291-9b91-40d9-87f7-4e38aeaecfd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:58.278733 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc109291-9b91-40d9-87f7-4e38aeaecfd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:58.279013 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc109291-9b91-40d9-87f7-4e38aeaecfd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:58.279286 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc109291-9b91-40d9-87f7-4e38aeaecfd6 admin admin] PATH_INFO: `/v3/users/a65be4eeb4054d79ab392bc723323339` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:58.280410 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc109291-9b91-40d9-87f7-4e38aeaecfd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a65be4eeb4054d79ab392bc723323339)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:58.282315 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc109291-9b91-40d9-87f7-4e38aeaecfd6 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:58.485507 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-cc109291-9b91-40d9-87f7-4e38aeaecfd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a65be4eeb4054d79ab392bc723323339'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:58.492208 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-cc109291-9b91-40d9-87f7-4e38aeaecfd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a65be4eeb4054d79ab392bc723323339'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:58.493669 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-cc109291-9b91-40d9-87f7-4e38aeaecfd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a65be4eeb4054d79ab392bc723323339'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:58.597497 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-cc109291-9b91-40d9-87f7-4e38aeaecfd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a65be4eeb4054d79ab392bc723323339'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:58.610314 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-cc109291-9b91-40d9-87f7-4e38aeaecfd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a65be4eeb4054d79ab392bc723323339'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:58.616191 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1448/2908] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 06:04:58 2022] DELETE /identity/v3/users/a65be4eeb4054d79ab392bc723323339 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:04:58.621284 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285120a2-3f4e-4258-81ad-27fda6abb1cd None None] REQUEST_METHOD: `POST` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:58.621770 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285120a2-3f4e-4258-81ad-27fda6abb1cd None None] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:58.621877 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285120a2-3f4e-4258-81ad-27fda6abb1cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:58.640670 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.auth.core [None req-285120a2-3f4e-4258-81ad-27fda6abb1cd None None] MFA Rules not processed for user `359799b7d001453b90d984a05215cce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81670) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:04:58.641883 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-285120a2-3f4e-4258-81ad-27fda6abb1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:58.700203 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1461/2909] 10.222.0.15 () {58 vars in 958 bytes} [Wed Aug 24 06:04:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:04:58.709902 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc109291-9b91-40d9-87f7-4e38aeaecfd6 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:58.711893 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-cc109291-9b91-40d9-87f7-4e38aeaecfd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:58.767981 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc109291-9b91-40d9-87f7-4e38aeaecfd6 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:58.771502 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380f9421-1796-4f13-aadc-2c6f62a13079 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:58.772093 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f9421-1796-4f13-aadc-2c6f62a13079 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:58.772371 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f9421-1796-4f13-aadc-2c6f62a13079 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:58.772676 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f9421-1796-4f13-aadc-2c6f62a13079 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:58.782275 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380f9421-1796-4f13-aadc-2c6f62a13079 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:58.783737 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380f9421-1796-4f13-aadc-2c6f62a13079 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:58.793831 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1449/2910] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:04:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:04:59.221213 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285120a2-3f4e-4258-81ad-27fda6abb1cd None None] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:59.222649 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-285120a2-3f4e-4258-81ad-27fda6abb1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:04:59.235939 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285120a2-3f4e-4258-81ad-27fda6abb1cd None None] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:59.241218 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9685ed3-2c14-4b06-bcd6-ec9841e09364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:59.241885 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9685ed3-2c14-4b06-bcd6-ec9841e09364 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:59.242170 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9685ed3-2c14-4b06-bcd6-ec9841e09364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:59.242999 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9685ed3-2c14-4b06-bcd6-ec9841e09364 admin admin] PATH_INFO: `/v3/projects/6a56c0475f3242d6aa22227c5c6d84ac` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:59.244629 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9685ed3-2c14-4b06-bcd6-ec9841e09364 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a56c0475f3242d6aa22227c5c6d84ac)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:59.246800 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9685ed3-2c14-4b06-bcd6-ec9841e09364 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:59.263262 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:04:59.372805 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-d9685ed3-2c14-4b06-bcd6-ec9841e09364 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a56c0475f3242d6aa22227c5c6d84ac', 'request_id': 'req-d9685ed3-2c14-4b06-bcd6-ec9841e09364'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:59.378851 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-d9685ed3-2c14-4b06-bcd6-ec9841e09364 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a56c0475f3242d6aa22227c5c6d84ac', 'request_id': 'req-d9685ed3-2c14-4b06-bcd6-ec9841e09364'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:59.460165 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-d9685ed3-2c14-4b06-bcd6-ec9841e09364 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a56c0475f3242d6aa22227c5c6d84ac', 'request_id': 'req-d9685ed3-2c14-4b06-bcd6-ec9841e09364'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:59.473416 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-d9685ed3-2c14-4b06-bcd6-ec9841e09364 admin admin] The token cache is being invalidate because project 6a56c0475f3242d6aa22227c5c6d84ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:04:59.473710 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-d9685ed3-2c14-4b06-bcd6-ec9841e09364 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:59.474370 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-d9685ed3-2c14-4b06-bcd6-ec9841e09364 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:59.476002 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1462/2911] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 06:04:59 2022] DELETE /identity/v3/projects/6a56c0475f3242d6aa22227c5c6d84ac => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:04:59.481343 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380f9421-1796-4f13-aadc-2c6f62a13079 service neutron] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:04:59.491101 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380f9421-1796-4f13-aadc-2c6f62a13079 service neutron] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:04:59.504003 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:04:59.547643 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9650815d-9d22-43b6-901e-c282eaf322b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:04:59.548258 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9650815d-9d22-43b6-901e-c282eaf322b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:04:59.548570 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9650815d-9d22-43b6-901e-c282eaf322b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:04:59.548844 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9650815d-9d22-43b6-901e-c282eaf322b0 admin admin] PATH_INFO: `/v3/users/a4e0eb045b684e8c97e35210891b6b15` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:04:59.549947 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9650815d-9d22-43b6-901e-c282eaf322b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4e0eb045b684e8c97e35210891b6b15)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:04:59.551853 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9650815d-9d22-43b6-901e-c282eaf322b0 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:04:59.760867 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-9650815d-9d22-43b6-901e-c282eaf322b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4e0eb045b684e8c97e35210891b6b15'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:59.767971 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-9650815d-9d22-43b6-901e-c282eaf322b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4e0eb045b684e8c97e35210891b6b15'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:59.768666 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-9650815d-9d22-43b6-901e-c282eaf322b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4e0eb045b684e8c97e35210891b6b15'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:59.847529 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-9650815d-9d22-43b6-901e-c282eaf322b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4e0eb045b684e8c97e35210891b6b15'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:59.860270 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-9650815d-9d22-43b6-901e-c282eaf322b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4e0eb045b684e8c97e35210891b6b15'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:04:59.866716 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1450/2912] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 06:04:59 2022] DELETE /identity/v3/users/a4e0eb045b684e8c97e35210891b6b15 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:05:00.319325 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9685ed3-2c14-4b06-bcd6-ec9841e09364 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:05:00.323881 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-d9685ed3-2c14-4b06-bcd6-ec9841e09364 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:05:00.334855 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9685ed3-2c14-4b06-bcd6-ec9841e09364 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:05:00.393190 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576829eb-920d-4807-81df-47e11ba95d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:05:00.393821 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576829eb-920d-4807-81df-47e11ba95d5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:05:00.394103 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576829eb-920d-4807-81df-47e11ba95d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:05:00.394381 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576829eb-920d-4807-81df-47e11ba95d5b admin admin] PATH_INFO: `/v3/projects/67e20a540d41490a85c35d05fdd1c202` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:05:00.395504 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576829eb-920d-4807-81df-47e11ba95d5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=67e20a540d41490a85c35d05fdd1c202)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:05:00.397463 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576829eb-920d-4807-81df-47e11ba95d5b admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:05:00.529403 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-576829eb-920d-4807-81df-47e11ba95d5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67e20a540d41490a85c35d05fdd1c202', 'request_id': 'req-576829eb-920d-4807-81df-47e11ba95d5b'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:05:00.535525 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-576829eb-920d-4807-81df-47e11ba95d5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67e20a540d41490a85c35d05fdd1c202', 'request_id': 'req-576829eb-920d-4807-81df-47e11ba95d5b'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:05:00.643176 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-576829eb-920d-4807-81df-47e11ba95d5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67e20a540d41490a85c35d05fdd1c202', 'request_id': 'req-576829eb-920d-4807-81df-47e11ba95d5b'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:05:00.654675 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-576829eb-920d-4807-81df-47e11ba95d5b admin admin] The token cache is being invalidate because project 67e20a540d41490a85c35d05fdd1c202 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81670) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:05:00.654987 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-576829eb-920d-4807-81df-47e11ba95d5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:05:00.655690 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-576829eb-920d-4807-81df-47e11ba95d5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:05:00.657195 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1463/2913] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 06:05:00 2022] DELETE /identity/v3/projects/67e20a540d41490a85c35d05fdd1c202 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:05:01.541717 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9650815d-9d22-43b6-901e-c282eaf322b0 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:05:01.543244 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-9650815d-9d22-43b6-901e-c282eaf322b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:05:01.603606 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9650815d-9d22-43b6-901e-c282eaf322b0 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:05:01.652578 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aabd91-78f8-4348-9079-d831e78b2dd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:05:01.653193 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aabd91-78f8-4348-9079-d831e78b2dd4 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:05:01.653477 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aabd91-78f8-4348-9079-d831e78b2dd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:05:01.653803 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aabd91-78f8-4348-9079-d831e78b2dd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:05:01.655579 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-34aabd91-78f8-4348-9079-d831e78b2dd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:05:01.659645 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:05:01.741128 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aabd91-78f8-4348-9079-d831e78b2dd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:05:01.742476 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aabd91-78f8-4348-9079-d831e78b2dd4 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:05:01.763727 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1451/2914] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:05:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:05:02.943679 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576829eb-920d-4807-81df-47e11ba95d5b admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:05:02.945278 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-576829eb-920d-4807-81df-47e11ba95d5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:05:02.958369 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576829eb-920d-4807-81df-47e11ba95d5b admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:05:02.962108 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8254c93-09a2-4eb1-bf62-2f8671520c4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:05:02.962705 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8254c93-09a2-4eb1-bf62-2f8671520c4c service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:05:02.963017 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8254c93-09a2-4eb1-bf62-2f8671520c4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:05:02.963295 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8254c93-09a2-4eb1-bf62-2f8671520c4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:05:02.966123 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-b8254c93-09a2-4eb1-bf62-2f8671520c4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:05:02.979263 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8254c93-09a2-4eb1-bf62-2f8671520c4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:05:02.981158 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8254c93-09a2-4eb1-bf62-2f8671520c4c service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:05:02.992235 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1464/2915] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:05:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:05:11.883527 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=42 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:05:12.987318 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=75,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:05:16.591218 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aabd91-78f8-4348-9079-d831e78b2dd4 service neutron] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:05:16.592780 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-34aabd91-78f8-4348-9079-d831e78b2dd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:05:16.603772 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aabd91-78f8-4348-9079-d831e78b2dd4 service neutron] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:05:16.607408 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dd3234-01e0-4d78-8f38-332d723bca25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:05:16.608044 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd3234-01e0-4d78-8f38-332d723bca25 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:05:16.608328 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd3234-01e0-4d78-8f38-332d723bca25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:05:16.608643 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd3234-01e0-4d78-8f38-332d723bca25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:05:16.610422 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-f6dd3234-01e0-4d78-8f38-332d723bca25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:05:16.620587 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dd3234-01e0-4d78-8f38-332d723bca25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:05:16.622019 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dd3234-01e0-4d78-8f38-332d723bca25 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:05:16.631054 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1452/2916] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:05:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:05:16.722973 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8254c93-09a2-4eb1-bf62-2f8671520c4c service neutron] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:05:16.731554 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8254c93-09a2-4eb1-bf62-2f8671520c4c service neutron] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:05:16.735689 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94ee8c1-a2d1-473a-abb6-a6085012032a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:05:16.736300 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ee8c1-a2d1-473a-abb6-a6085012032a service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:05:16.736606 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ee8c1-a2d1-473a-abb6-a6085012032a service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:05:16.736877 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ee8c1-a2d1-473a-abb6-a6085012032a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:05:16.738734 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-c94ee8c1-a2d1-473a-abb6-a6085012032a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:05:16.753704 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94ee8c1-a2d1-473a-abb6-a6085012032a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:05:16.755218 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94ee8c1-a2d1-473a-abb6-a6085012032a service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:05:16.767857 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1465/2917] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:05:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:05:26.627327 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=6 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:05:26.761561 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=6 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:05:56.634538 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dd3234-01e0-4d78-8f38-332d723bca25 service neutron] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:05:56.643558 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dd3234-01e0-4d78-8f38-332d723bca25 service neutron] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:05:56.647255 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd6623-eafa-4571-9e89-7814d67d7458 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:05:56.648454 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd6623-eafa-4571-9e89-7814d67d7458 service neutron] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:05:56.648742 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd6623-eafa-4571-9e89-7814d67d7458 service neutron] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:05:56.649016 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd6623-eafa-4571-9e89-7814d67d7458 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:05:56.650791 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-9efd6623-eafa-4571-9e89-7814d67d7458 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:05:56.660690 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd6623-eafa-4571-9e89-7814d67d7458 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:05:56.662106 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd6623-eafa-4571-9e89-7814d67d7458 service neutron] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:05:56.670962 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1453/2918] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:05:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:06:06.667133 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=6 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:07:12.181800 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94ee8c1-a2d1-473a-abb6-a6085012032a service neutron] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:07:12.190597 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94ee8c1-a2d1-473a-abb6-a6085012032a service neutron] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:07:12.194802 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0fb626-03e7-4025-b16b-53e750f1dfd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:07:12.195406 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0fb626-03e7-4025-b16b-53e750f1dfd3 service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:07:12.195694 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0fb626-03e7-4025-b16b-53e750f1dfd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:07:12.195975 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0fb626-03e7-4025-b16b-53e750f1dfd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:07:12.197764 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-8a0fb626-03e7-4025-b16b-53e750f1dfd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:07:12.257784 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0fb626-03e7-4025-b16b-53e750f1dfd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:07:12.259251 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0fb626-03e7-4025-b16b-53e750f1dfd3 service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:07:12.268160 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1466/2919] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:07:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:07:13.463317 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd6623-eafa-4571-9e89-7814d67d7458 service neutron] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:07:13.465467 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-9efd6623-eafa-4571-9e89-7814d67d7458 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:07:13.483226 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd6623-eafa-4571-9e89-7814d67d7458 service neutron] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:07:13.488570 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f387bdbd-b462-4dc6-b849-3f8a51350a90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbecffb800584c5786813accd061421b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:07:13.489832 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387bdbd-b462-4dc6-b849-3f8a51350a90 service nova] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:07:13.490328 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387bdbd-b462-4dc6-b849-3f8a51350a90 service nova] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:07:13.490770 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387bdbd-b462-4dc6-b849-3f8a51350a90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:07:13.492933 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-f387bdbd-b462-4dc6-b849-3f8a51350a90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:07:13.505851 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f387bdbd-b462-4dc6-b849-3f8a51350a90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:07:13.509012 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f387bdbd-b462-4dc6-b849-3f8a51350a90 service nova] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:07:13.520017 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1454/2920] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:07:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:07:18.986377 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0fb626-03e7-4025-b16b-53e750f1dfd3 service neutron] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:07:18.995724 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0fb626-03e7-4025-b16b-53e750f1dfd3 service neutron] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:07:18.999341 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978f5aa-e5fe-476d-b7f4-345a342bda7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ecd9b302b64ecf81d7e3d3356ddf7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:07:18.999939 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978f5aa-e5fe-476d-b7f4-345a342bda7d service neutron] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:07:19.000221 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978f5aa-e5fe-476d-b7f4-345a342bda7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:07:19.000534 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978f5aa-e5fe-476d-b7f4-345a342bda7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:07:19.002262 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-7978f5aa-e5fe-476d-b7f4-345a342bda7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:07:19.070654 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978f5aa-e5fe-476d-b7f4-345a342bda7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:07:19.072840 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978f5aa-e5fe-476d-b7f4-345a342bda7d service neutron] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:07:19.082776 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1467/2921] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:07:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:07:22.633881 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f387bdbd-b462-4dc6-b849-3f8a51350a90 service nova] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:07:22.639430 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-f387bdbd-b462-4dc6-b849-3f8a51350a90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:07:22.715033 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f387bdbd-b462-4dc6-b849-3f8a51350a90 service nova] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:07:22.720137 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e6c3a-6fa0-4dbe-8c9a-353de5f4e6f1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6778b7c5ad19484b877494f143050405', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:07:22.725576 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e6c3a-6fa0-4dbe-8c9a-353de5f4e6f1 service placement] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:07:22.725576 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e6c3a-6fa0-4dbe-8c9a-353de5f4e6f1 service placement] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:07:22.725576 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e6c3a-6fa0-4dbe-8c9a-353de5f4e6f1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:07:22.725576 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-f59e6c3a-6fa0-4dbe-8c9a-353de5f4e6f1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:07:22.735416 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59e6c3a-6fa0-4dbe-8c9a-353de5f4e6f1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:07:22.737129 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59e6c3a-6fa0-4dbe-8c9a-353de5f4e6f1 service placement] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:07:22.747657 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1455/2922] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:07:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:07:26.397136 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978f5aa-e5fe-476d-b7f4-345a342bda7d service neutron] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:07:26.398603 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-7978f5aa-e5fe-476d-b7f4-345a342bda7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:07:26.411434 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978f5aa-e5fe-476d-b7f4-345a342bda7d service neutron] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:07:26.416193 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59511712-1241-40c8-91b5-f8750332cf74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:07:26.416859 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59511712-1241-40c8-91b5-f8750332cf74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:07:26.417138 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59511712-1241-40c8-91b5-f8750332cf74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:07:26.417411 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59511712-1241-40c8-91b5-f8750332cf74 admin admin] PATH_INFO: `/v3/users/3b4870c15b8149908b6c1a99b1f73bc0` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:07:26.418524 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59511712-1241-40c8-91b5-f8750332cf74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b4870c15b8149908b6c1a99b1f73bc0)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:07:26.420447 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59511712-1241-40c8-91b5-f8750332cf74 admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:07:26.574315 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-59511712-1241-40c8-91b5-f8750332cf74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b4870c15b8149908b6c1a99b1f73bc0'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:07:26.635524 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-59511712-1241-40c8-91b5-f8750332cf74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b4870c15b8149908b6c1a99b1f73bc0'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:07:26.643070 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-59511712-1241-40c8-91b5-f8750332cf74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b4870c15b8149908b6c1a99b1f73bc0'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:07:26.656268 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-59511712-1241-40c8-91b5-f8750332cf74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b4870c15b8149908b6c1a99b1f73bc0'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:07:26.657148 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-59511712-1241-40c8-91b5-f8750332cf74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b4870c15b8149908b6c1a99b1f73bc0'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:07:26.663213 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1468/2923] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 06:07:26 2022] DELETE /identity/v3/users/3b4870c15b8149908b6c1a99b1f73bc0 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:07:26.952656 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e6c3a-6fa0-4dbe-8c9a-353de5f4e6f1 service placement] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:07:26.964791 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e6c3a-6fa0-4dbe-8c9a-353de5f4e6f1 service placement] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:07:27.022404 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef4bc2-173f-41b2-ba41-398c37e56ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:07:27.023035 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef4bc2-173f-41b2-ba41-398c37e56ddf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:07:27.023317 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef4bc2-173f-41b2-ba41-398c37e56ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:07:27.023592 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef4bc2-173f-41b2-ba41-398c37e56ddf admin admin] PATH_INFO: `/v3/projects/7373b9ca241b4fbba12f2413458072bb` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:07:27.025565 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ef4bc2-173f-41b2-ba41-398c37e56ddf admin admin] RBAC: Authorizing `identity:delete_project(project_id=7373b9ca241b4fbba12f2413458072bb)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:07:27.027470 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ef4bc2-173f-41b2-ba41-398c37e56ddf admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:07:27.207719 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-34ef4bc2-173f-41b2-ba41-398c37e56ddf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7373b9ca241b4fbba12f2413458072bb', 'request_id': 'req-34ef4bc2-173f-41b2-ba41-398c37e56ddf'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:07:27.214236 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-34ef4bc2-173f-41b2-ba41-398c37e56ddf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7373b9ca241b4fbba12f2413458072bb', 'request_id': 'req-34ef4bc2-173f-41b2-ba41-398c37e56ddf'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:07:27.315562 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-34ef4bc2-173f-41b2-ba41-398c37e56ddf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7373b9ca241b4fbba12f2413458072bb', 'request_id': 'req-34ef4bc2-173f-41b2-ba41-398c37e56ddf'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:07:27.326962 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-34ef4bc2-173f-41b2-ba41-398c37e56ddf admin admin] The token cache is being invalidate because project 7373b9ca241b4fbba12f2413458072bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:07:27.327253 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-34ef4bc2-173f-41b2-ba41-398c37e56ddf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:07:27.327885 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-34ef4bc2-173f-41b2-ba41-398c37e56ddf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:07:27.329405 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1456/2924] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 06:07:26 2022] DELETE /identity/v3/projects/7373b9ca241b4fbba12f2413458072bb => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:07:27.333385 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59511712-1241-40c8-91b5-f8750332cf74 admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:07:27.334858 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-59511712-1241-40c8-91b5-f8750332cf74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:07:27.345984 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:07:27.349808 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59511712-1241-40c8-91b5-f8750332cf74 admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:07:27.406621 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ac649-7857-4bf0-bc05-f02d584cf39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:07:27.407304 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ac649-7857-4bf0-bc05-f02d584cf39c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:07:27.407533 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ac649-7857-4bf0-bc05-f02d584cf39c admin admin] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:07:27.407804 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ac649-7857-4bf0-bc05-f02d584cf39c admin admin] PATH_INFO: `/v3/users/4cc78766dcb34809b4a7fdbead0335de` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:07:27.408956 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163ac649-7857-4bf0-bc05-f02d584cf39c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cc78766dcb34809b4a7fdbead0335de)` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:07:27.412157 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163ac649-7857-4bf0-bc05-f02d584cf39c admin admin] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:07:27.614971 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-163ac649-7857-4bf0-bc05-f02d584cf39c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cc78766dcb34809b4a7fdbead0335de'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:07:27.715407 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-163ac649-7857-4bf0-bc05-f02d584cf39c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cc78766dcb34809b4a7fdbead0335de'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:07:27.722592 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-163ac649-7857-4bf0-bc05-f02d584cf39c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cc78766dcb34809b4a7fdbead0335de'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:07:27.734650 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-163ac649-7857-4bf0-bc05-f02d584cf39c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cc78766dcb34809b4a7fdbead0335de'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:07:27.735350 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.notifications [None req-163ac649-7857-4bf0-bc05-f02d584cf39c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cc78766dcb34809b4a7fdbead0335de'} {{(pid=81670) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:07:27.741322 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1469/2925] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Aug 24 06:07:27 2022] DELETE /identity/v3/users/4cc78766dcb34809b4a7fdbead0335de => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:07:27.991063 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef4bc2-173f-41b2-ba41-398c37e56ddf admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:07:27.993306 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-34ef4bc2-173f-41b2-ba41-398c37e56ddf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:07:28.006572 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef4bc2-173f-41b2-ba41-398c37e56ddf admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:07:28.011175 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:07:28.063837 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34835cf2-2953-43fd-8c9d-4e99ca9c3ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359799b7d001453b90d984a05215cce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '617e47339cb94e20967707fe62db290d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:07:28.064510 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34835cf2-2953-43fd-8c9d-4e99ca9c3ce4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:07:28.064791 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34835cf2-2953-43fd-8c9d-4e99ca9c3ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:07:28.065070 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34835cf2-2953-43fd-8c9d-4e99ca9c3ce4 admin admin] PATH_INFO: `/v3/projects/10c79dadf1674df8b89441d26a09dc87` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:07:28.066198 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34835cf2-2953-43fd-8c9d-4e99ca9c3ce4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10c79dadf1674df8b89441d26a09dc87)` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:07:28.068105 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34835cf2-2953-43fd-8c9d-4e99ca9c3ce4 admin admin] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:07:28.217791 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-34835cf2-2953-43fd-8c9d-4e99ca9c3ce4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10c79dadf1674df8b89441d26a09dc87', 'request_id': 'req-34835cf2-2953-43fd-8c9d-4e99ca9c3ce4'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:07:28.223955 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-34835cf2-2953-43fd-8c9d-4e99ca9c3ce4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10c79dadf1674df8b89441d26a09dc87', 'request_id': 'req-34835cf2-2953-43fd-8c9d-4e99ca9c3ce4'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:07:28.296545 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-34835cf2-2953-43fd-8c9d-4e99ca9c3ce4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10c79dadf1674df8b89441d26a09dc87', 'request_id': 'req-34835cf2-2953-43fd-8c9d-4e99ca9c3ce4'} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:07:28.307415 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-34835cf2-2953-43fd-8c9d-4e99ca9c3ce4 admin admin] The token cache is being invalidate because project 10c79dadf1674df8b89441d26a09dc87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81671) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:07:28.307706 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-34835cf2-2953-43fd-8c9d-4e99ca9c3ce4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:07:28.308334 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.notifications [None req-34835cf2-2953-43fd-8c9d-4e99ca9c3ce4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81671) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:07:28.309845 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1457/2926] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Aug 24 06:07:27 2022] DELETE /identity/v3/projects/10c79dadf1674df8b89441d26a09dc87 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:07:37.737816 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=51,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:07:38.339690 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:07:58.409445 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ac649-7857-4bf0-bc05-f02d584cf39c admin admin] Authenticating user token {{(pid=81670) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:07:58.410869 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-163ac649-7857-4bf0-bc05-f02d584cf39c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:07:58.482246 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ac649-7857-4bf0-bc05-f02d584cf39c admin admin] Validating token access rules against request {{(pid=81670) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:07:58.542141 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8681da4e-e9c9-44bc-aadf-fb15cc0bfc88 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6778b7c5ad19484b877494f143050405', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81670) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:07:58.542780 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681da4e-e9c9-44bc-aadf-fb15cc0bfc88 service placement] REQUEST_METHOD: `GET` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:07:58.543062 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681da4e-e9c9-44bc-aadf-fb15cc0bfc88 service placement] SCRIPT_NAME: `/identity` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:07:58.543340 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681da4e-e9c9-44bc-aadf-fb15cc0bfc88 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81670) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:07:58.545146 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.fernet_utils [None req-8681da4e-e9c9-44bc-aadf-fb15cc0bfc88 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81670) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:07:58.556680 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8681da4e-e9c9-44bc-aadf-fb15cc0bfc88 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:07:58.558261 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8681da4e-e9c9-44bc-aadf-fb15cc0bfc88 service placement] RBAC: Authorization granted {{(pid=81670) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:07:58.568050 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: [pid: 81670|app: 0|req: 1470/2927] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:07:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:08:08.563928 dv-cinder-854267-1-iscsi devstack@keystone.service[81670]: DEBUG dbcounter [-] [81670] Writing DB stats keystone:SELECT=58 {{(pid=81670) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:13:03.213354 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34835cf2-2953-43fd-8c9d-4e99ca9c3ce4 admin admin] Authenticating user token {{(pid=81671) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:13:03.214871 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.fernet_utils [None req-34835cf2-2953-43fd-8c9d-4e99ca9c3ce4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81671) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:13:03.285202 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34835cf2-2953-43fd-8c9d-4e99ca9c3ce4 admin admin] Validating token access rules against request {{(pid=81671) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:13:03.293982 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede2de0-79dc-4e66-936e-62df48ea1ec2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6778b7c5ad19484b877494f143050405', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59592706ae9442e08353158c625e9581', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81671) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:13:03.294615 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede2de0-79dc-4e66-936e-62df48ea1ec2 service placement] REQUEST_METHOD: `GET` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:13:03.294898 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede2de0-79dc-4e66-936e-62df48ea1ec2 service placement] SCRIPT_NAME: `/identity` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:13:03.295630 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede2de0-79dc-4e66-936e-62df48ea1ec2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81671) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:13:03.307983 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ede2de0-79dc-4e66-936e-62df48ea1ec2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:13:03.309919 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ede2de0-79dc-4e66-936e-62df48ea1ec2 service placement] RBAC: Authorization granted {{(pid=81671) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:13:03.320809 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: [pid: 81671|app: 0|req: 1458/2928] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Aug 24 06:13:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:13:13.316170 dv-cinder-854267-1-iscsi devstack@keystone.service[81671]: DEBUG dbcounter [-] [81671] Writing DB stats keystone:SELECT=24 {{(pid=81671) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}